<!-- build time:Wed Jun 21 2023 22:33:34 GMT+0800 (GMT+08:00) --><!DOCTYPE html><html lang="zh-CN"><head><meta charset="UTF-8"><meta name="viewport" content="width=device-width,initial-scale=1,maximum-scale=2"><meta name="theme-color" content="#FFF"><meta name="baidu-site-verification" content="code-C0oocRvMWv"><link rel="apple-touch-icon" sizes="180x180" href="/images/apple-touch-icon.png"><link rel="icon" type="image/ico" sizes="32x32" href="/images/favicon.ico"><link rel="mask-icon" href="/images/logo.svg" color=""><link rel="manifest" href="/images/manifest.json"><meta name="msapplication-config" content="/images/browserconfig.xml"><meta http-equiv="Cache-Control" content="no-transform"><meta http-equiv="Cache-Control" content="no-siteapp"><meta name="baidu-site-verification" content="https://jiang-hs.gitee.io"><link rel="alternate" type="application/rss+xml" title="航 順" href="https://jiang-hs.gitee.io/rss.xml"><link rel="alternate" type="application/atom+xml" title="航 順" href="https://jiang-hs.gitee.io/atom.xml"><link rel="alternate" type="application/json" title="航 順" href="https://jiang-hs.gitee.io/feed.json"><link rel="stylesheet" href="//fonts.googleapis.com/css?family=Mulish:300,300italic,400,400italic,700,700italic%7CFredericka%20the%20Great:300,300italic,400,400italic,700,700italic%7CNoto%20Serif%20JP:300,300italic,400,400italic,700,700italic%7CNoto%20Serif%20SC:300,300italic,400,400italic,700,700italic%7CInconsolata:300,300italic,400,400italic,700,700italic&display=swap&subset=latin,latin-ext"><link rel="stylesheet" href="/css/app.css?v=0.0.0"><link rel="canonical" href="https://jiang-hs.gitee.io/posts/b3064b8/"><meta name="description" content="# 1 一元线性回归 # 1.1 什么是回归分析 回归分析是一种预测性的建模技术，它研究的是因变量（目标）和自变量（预测器）之间的关系。这种技术通常用于预测分析，时间序列模型以及发现变量之间的因果关系。通常使用曲线 &#x2F; 线来拟合数据点，目标是使曲线到数据点的距离差异最小。 # 1.2 线性回归 线性回归是回归问题中的一种，线性回归假设目标值与特征之间线性相关，即满足一个多元一次方程。 如果只包括一"><meta property="og:type" content="article"><meta property="og:title" content="梯度下降及线性回归详解"><meta property="og:url" content="https://jiang-hs.gitee.io/posts/b3064b8/index.html"><meta property="og:site_name" content="航 順"><meta property="og:description" content="# 1 一元线性回归 # 1.1 什么是回归分析 回归分析是一种预测性的建模技术，它研究的是因变量（目标）和自变量（预测器）之间的关系。这种技术通常用于预测分析，时间序列模型以及发现变量之间的因果关系。通常使用曲线 &#x2F; 线来拟合数据点，目标是使曲线到数据点的距离差异最小。 # 1.2 线性回归 线性回归是回归问题中的一种，线性回归假设目标值与特征之间线性相关，即满足一个多元一次方程。 如果只包括一"><meta property="og:locale" content="zh_CN"><meta property="og:image" content="https://shun309.oss-cn-hangzhou.aliyuncs.com/photos/856725-20190303225317608-462127381.png"><meta property="og:image" content="https://shun309.oss-cn-hangzhou.aliyuncs.com/photos/856725-20190303225403953-1601026207.png"><meta property="og:image" content="https://shun309.oss-cn-hangzhou.aliyuncs.com/photos/856725-20190303225443234-242375165.png"><meta property="og:image" content="https://shun309.oss-cn-hangzhou.aliyuncs.com/photos/856725-20190303225522497-461663601.png"><meta property="og:image" content="https://shun309.oss-cn-hangzhou.aliyuncs.com/photos/856725-20190303225705480-236089.png"><meta property="og:image" content="https://pic2.zhimg.com/50/v2-b722c2fca0ea2c1bc71975dd965d0c97_hd.webp?source=1940ef5c"><meta property="og:image" content="https://shun309.oss-cn-hangzhou.aliyuncs.com/photos/v2-aa03253883db6239bbdaa9db3ee6fcf5_hd.webp"><meta property="og:image" content="https://shun309.oss-cn-hangzhou.aliyuncs.com/photos/v2-3c9538a23d6f453a698120c124bfa76e_hd.webp"><meta property="og:image" content="https://shun309.oss-cn-hangzhou.aliyuncs.com/photos/v2-be1591ca247ab8485e13bb9725cb6f32_hd.webp"><meta property="og:image" content="https://shun309.oss-cn-hangzhou.aliyuncs.com/photos/v2-5602652d76fc62b94fc7424ae87ba955_hd.webp"><meta property="og:image" content="https://shun309.oss-cn-hangzhou.aliyuncs.com/photos/20210415215318.png"><meta property="og:image" content="https://shun309.oss-cn-hangzhou.aliyuncs.com/photos/20210415223448.png"><meta property="og:image" content="https://shun309.oss-cn-hangzhou.aliyuncs.com/photos/20210415223533.png"><meta property="og:image" content="https://shun309.oss-cn-hangzhou.aliyuncs.com/photos/20210415223635.png"><meta property="og:image" content="https://shun309.oss-cn-hangzhou.aliyuncs.com/photos/20210415230137.png"><meta property="og:image" content="https://shun309.oss-cn-hangzhou.aliyuncs.com/photos/20210415231826.png"><meta property="og:image" content="https://shun309.oss-cn-hangzhou.aliyuncs.com/photos/20210415232202.png"><meta property="og:image" content="https://jiang-hs.github.io/post-images/1594871083036.jpg"><meta property="og:image" content="https://shun309.oss-cn-hangzhou.aliyuncs.com/photos/20210415234133.png"><meta property="og:image" content="https://shun309.oss-cn-hangzhou.aliyuncs.com/photos/20210416093445.png"><meta property="og:image" content="https://shun309.oss-cn-hangzhou.aliyuncs.com/photos/20210416093606.png"><meta property="article:published_time" content="2021-04-16T01:40:08.000Z"><meta property="article:modified_time" content="2023-04-19T03:19:53.097Z"><meta property="article:author" content="hang shun"><meta property="article:tag" content="hang shun"><meta name="twitter:card" content="summary"><meta name="twitter:image" content="https://shun309.oss-cn-hangzhou.aliyuncs.com/photos/856725-20190303225317608-462127381.png"><title>梯度下降及线性回归详解 | hang shun = 航 順 = 天官赐福，百无禁忌</title><meta name="generator" content="Hexo 5.4.2"></head><body itemscope itemtype="http://schema.org/WebPage"><div id="loading"><div class="cat"><div class="body"></div><div class="head"><div class="face"></div></div><div class="foot"><div class="tummy-end"></div><div class="bottom"></div><div class="legs left"></div><div class="legs right"></div></div><div class="paw"><div class="hands left"></div><div class="hands right"></div></div></div></div><div id="container"><header id="header" itemscope itemtype="http://schema.org/WPHeader"><div class="inner"><div id="brand"><div class="pjax"><h1 itemprop="name headline">梯度下降及线性回归详解</h1><div class="meta"><span class="item" title="创建时间：2021-04-16 09:40:08"><span class="icon"><i class="ic i-calendar"></i> </span><span class="text">发表于</span> <time itemprop="dateCreated datePublished" datetime="2021-04-16T09:40:08+08:00">2021-04-16</time> </span><span class="item" title="本文字数"><span class="icon"><i class="ic i-pen"></i> </span><span class="text">本文字数</span> <span>12k</span> <span class="text">字</span> </span><span class="item" title="阅读时长"><span class="icon"><i class="ic i-clock"></i> </span><span class="text">阅读时长</span> <span>11 分钟</span></span></div></div></div><nav id="nav"><div class="inner"><div class="toggle"><div class="lines" aria-label="切换导航栏"><span class="line"></span> <span class="line"></span> <span class="line"></span></div></div><ul class="menu"><li class="item title"><a href="/" rel="start">hang shun</a></li></ul><ul class="right"><li class="item theme"><i class="ic i-sun"></i></li><li class="item search"><i class="ic i-search"></i></li></ul></div></nav></div><div id="imgs" class="pjax"><ul><li class="item" data-background-image="https://pic1.imgdb.cn/item/64427a860d2dde5777acb22d.jpg"></li><li class="item" data-background-image="https://pic1.imgdb.cn/item/60d7f98f5132923bf8aa2419.jpg"></li><li class="item" data-background-image="https://pic1.imgdb.cn/item/60d7f9bf5132923bf8ab1edf.jpg"></li><li class="item" data-background-image="https://pic1.imgdb.cn/item/60d7f9855132923bf8a9f1d4.jpg"></li><li class="item" data-background-image="https://pic1.imgdb.cn/item/64427a840d2dde5777acafa6.jpg"></li><li class="item" data-background-image="https://pic1.imgdb.cn/item/60d7f98e5132923bf8aa237d.jpg"></li></ul></div></header><div id="waves"><svg class="waves" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" viewBox="0 24 150 28" preserveAspectRatio="none" shape-rendering="auto"><defs><path id="gentle-wave" d="M-160 44c30 0 58-18 88-18s 58 18 88 18 58-18 88-18 58 18 88 18 v44h-352z"/></defs><g class="parallax"><use xlink:href="#gentle-wave" x="48" y="0"/><use xlink:href="#gentle-wave" x="48" y="3"/><use xlink:href="#gentle-wave" x="48" y="5"/><use xlink:href="#gentle-wave" x="48" y="7"/></g></svg></div><main><div class="inner"><div id="main" class="pjax"><div class="article wrap"><div class="breadcrumb" itemscope itemtype="https://schema.org/BreadcrumbList"><i class="ic i-home"></i> <span><a href="/">首页</a></span></div><article itemscope itemtype="http://schema.org/Article" class="post block" lang="zh-CN"><link itemprop="mainEntityOfPage" href="https://jiang-hs.gitee.io/posts/b3064b8/"><span hidden itemprop="author" itemscope itemtype="http://schema.org/Person"><meta itemprop="image" content="/images/avatar.jpg"><meta itemprop="name" content="hang shun"><meta itemprop="description" content="天官赐福，百无禁忌, 世中逢尔，雨中逢花"></span><span hidden itemprop="publisher" itemscope itemtype="http://schema.org/Organization"><meta itemprop="name" content="航 順"></span><div class="body md" itemprop="articleBody"><h1 id="1-一元线性回归"><a class="anchor" href="#1-一元线性回归">#</a> 1 一元线性回归</h1><h2 id="11-什么是回归分析"><a class="anchor" href="#11-什么是回归分析">#</a> 1.1 什么是回归分析</h2><p>回归分析是一种预测性的建模技术，<strong>它研究的是因变量（目标）和自变量（预测器）之间的关系</strong>。这种技术通常用于预测分析，时间序列模型以及发现变量之间的因果关系。通常使用曲线 / 线来拟合数据点，目标是使曲线到数据点的距离差异最小。</p><h2 id="12-线性回归"><a class="anchor" href="#12-线性回归">#</a> 1.2 线性回归</h2><p>线性回归是回归问题中的一种，线性回归假设目标值与特征之间线性相关，即满足一个<strong>多元一次方程</strong>。</p><p>如果只包括一个自变量和一个因变量，且二者的关系可用一条直线近似表示，这种回归分析称为<strong>一元线性回归分析</strong>；如果回归分析中包括两个或两个以上的自变量，且因变量和自变量之间是线性关系，则称为<strong>多元线性回归分析</strong>。</p><p>通过构建损失函数，来求解损失函数最小时的参数<span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>w</mi></mrow><annotation encoding="application/x-tex">w</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:.43056em;vertical-align:0"></span><span class="mord mathnormal" style="margin-right:.02691em">w</span></span></span></span> 和<span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>b</mi></mrow><annotation encoding="application/x-tex">b</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:.69444em;vertical-align:0"></span><span class="mord mathnormal">b</span></span></span></span>。一元线性回归通长我们可以表达成如下公式：</p><p><img data-src="https://shun309.oss-cn-hangzhou.aliyuncs.com/photos/856725-20190303225317608-462127381.png" alt=""></p><p><span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mover accent="true"><mi>y</mi><mo>^</mo></mover></mrow><annotation encoding="application/x-tex">\hat{y}</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:.8888799999999999em;vertical-align:-.19444em"></span><span class="mord accent"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:.69444em"><span style="top:-3em"><span class="pstrut" style="height:3em"></span><span class="mord"><span class="mord mathnormal" style="margin-right:.03588em">y</span></span></span><span style="top:-3em"><span class="pstrut" style="height:3em"></span><span class="accent-body" style="left:-.19444em"><span class="mord">^</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.19444em"><span></span></span></span></span></span></span></span></span> 为预测值，自变量<span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>x</mi></mrow><annotation encoding="application/x-tex">x</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:.43056em;vertical-align:0"></span><span class="mord mathnormal">x</span></span></span></span> 和因变量<span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>y</mi></mrow><annotation encoding="application/x-tex">y</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:.625em;vertical-align:-.19444em"></span><span class="mord mathnormal" style="margin-right:.03588em">y</span></span></span></span> 是已知的，而我们想实现的是预测新增一个<span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>x</mi></mrow><annotation encoding="application/x-tex">x</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:.43056em;vertical-align:0"></span><span class="mord mathnormal">x</span></span></span></span>，其对应的<span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>y</mi></mrow><annotation encoding="application/x-tex">y</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:.625em;vertical-align:-.19444em"></span><span class="mord mathnormal" style="margin-right:.03588em">y</span></span></span></span> 是多少。因此，为了构建这个函数关系，目标是通过已知数据点，求解线性模型中<span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>w</mi></mrow><annotation encoding="application/x-tex">w</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:.43056em;vertical-align:0"></span><span class="mord mathnormal" style="margin-right:.02691em">w</span></span></span></span> 和<span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>b</mi></mrow><annotation encoding="application/x-tex">b</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:.69444em;vertical-align:0"></span><span class="mord mathnormal">b</span></span></span></span> 两个参数。</p><h2 id="13-目标损失函数"><a class="anchor" href="#13-目标损失函数">#</a> 1.3 目标 / 损失函数</h2><p><strong>损失函数</strong>用来评价模型的<strong>预测值</strong>和<strong>真实值</strong>不一样的程度，损失函数越好，通常模型的性能越好。不同的模型用的损失函数一般也不一样。在应用中，通常通过最小化损失函数求解和评估模型。</p><p>求解最佳参数，需要一个标准来对结果进行衡量，为此我们需要定量化一个目标函数式，使得计算机可以在求解过程中不断地优化。</p><p>针对任何模型求解问题，都是最终都是可以得到一组预测值<span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mover accent="true"><mi>y</mi><mo>^</mo></mover></mrow><annotation encoding="application/x-tex">\hat{y}</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:.8888799999999999em;vertical-align:-.19444em"></span><span class="mord accent"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:.69444em"><span style="top:-3em"><span class="pstrut" style="height:3em"></span><span class="mord"><span class="mord mathnormal" style="margin-right:.03588em">y</span></span></span><span style="top:-3em"><span class="pstrut" style="height:3em"></span><span class="accent-body" style="left:-.19444em"><span class="mord">^</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.19444em"><span></span></span></span></span></span></span></span></span>，对比已有的真实值 <span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>y</mi></mrow><annotation encoding="application/x-tex">y</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:.625em;vertical-align:-.19444em"></span><span class="mord mathnormal" style="margin-right:.03588em">y</span></span></span></span> ，数据行数为 $n $，可以将损失函数定义如下：</p><p><img data-src="https://shun309.oss-cn-hangzhou.aliyuncs.com/photos/856725-20190303225403953-1601026207.png" alt=""></p><p>即预测值与真实值之间的平均的平方距离，统计中一般称其为<strong> MAE (mean square error) 均方误差</strong>。把之前的函数式代入损失函数，并且将需要求解的参数 w 和 b 看做是函数 L 的自变量，可得：</p><p><img data-src="https://shun309.oss-cn-hangzhou.aliyuncs.com/photos/856725-20190303225443234-242375165.png" alt=""></p><p>现在的任务是求解最小化<span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>L</mi></mrow><annotation encoding="application/x-tex">L</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:.68333em;vertical-align:0"></span><span class="mord mathnormal">L</span></span></span></span> 时<span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>w</mi></mrow><annotation encoding="application/x-tex">w</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:.43056em;vertical-align:0"></span><span class="mord mathnormal" style="margin-right:.02691em">w</span></span></span></span> 和<span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>b</mi></mrow><annotation encoding="application/x-tex">b</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:.69444em;vertical-align:0"></span><span class="mord mathnormal">b</span></span></span></span> 的值，</p><p>即核心目标优化式为：</p><p><img data-src="https://shun309.oss-cn-hangzhou.aliyuncs.com/photos/856725-20190303225522497-461663601.png" alt=""></p><h2 id="两种求解方式"><a class="anchor" href="#两种求解方式">#</a> 两种求解方式：</h2><p>1）最小二乘法 (least square method)</p><p>这里不重点讲，可以自行了解。</p><p>2）梯度下降 (gradient descent,GD)</p><p>梯度下降核心内容是对自变量进行不断的更新（针对 w 和 b 求偏导），使得目标函数不断逼近最小值的过程</p><p><img data-src="https://shun309.oss-cn-hangzhou.aliyuncs.com/photos/856725-20190303225705480-236089.png" alt=""></p><h1 id="2-梯度下降gd"><a class="anchor" href="#2-梯度下降gd">#</a> 2 梯度下降 (GD)</h1><h2 id="21-梯度下降简介"><a class="anchor" href="#21-梯度下降简介">#</a> 2.1 梯度下降简介</h2><p>梯度下降法（gradient descent）是一个最优化算法，也称为最速下降法（steepest descent）。常用于机器学习和人工智能当中用来递归性地逼近最小偏差模型。寻找极小点时，从负梯度的方向寻找。当变化值小于一定阈值时或者完成一定次数时，梯度下降算法任务结束。</p><p><img data-src="https://pic2.zhimg.com/50/v2-b722c2fca0ea2c1bc71975dd965d0c97_hd.webp?source=1940ef5c" alt="img"></p><h2 id="22-核心公式"><a class="anchor" href="#22-核心公式">#</a> 2.2 核心公式</h2><p><span class="katex-display"><span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML" display="block"><semantics><mrow><msub><mi>θ</mi><mi>i</mi></msub><mo>=</mo><msub><mi>θ</mi><mi>i</mi></msub><mtext>−</mtext><mi>α</mi><mfrac><mi mathvariant="normal">∂</mi><mrow><mi mathvariant="normal">∂</mi><msub><mi>θ</mi><mi>i</mi></msub></mrow></mfrac><mi>J</mi><mo stretchy="false">(</mo><mi>θ</mi><mo stretchy="false">)</mo></mrow><annotation encoding="application/x-tex">θ_i=θ_i−α\frac{∂}{∂θ_i}J(θ)</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:.84444em;vertical-align:-.15em"></span><span class="mord"><span class="mord mathnormal" style="margin-right:.02778em">θ</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:.31166399999999994em"><span style="top:-2.5500000000000003em;margin-left:-.02778em;margin-right:.05em"><span class="pstrut" style="height:2.7em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight">i</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.15em"><span></span></span></span></span></span></span><span class="mspace" style="margin-right:.2777777777777778em"></span><span class="mrel">=</span><span class="mspace" style="margin-right:.2777777777777778em"></span></span><span class="base"><span class="strut" style="height:2.20744em;vertical-align:-.8360000000000001em"></span><span class="mord"><span class="mord mathnormal" style="margin-right:.02778em">θ</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:.31166399999999994em"><span style="top:-2.5500000000000003em;margin-left:-.02778em;margin-right:.05em"><span class="pstrut" style="height:2.7em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight">i</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.15em"><span></span></span></span></span></span></span><span class="mord">−</span><span class="mord mathnormal" style="margin-right:.0037em">α</span><span class="mord"><span class="mopen nulldelimiter"></span><span class="mfrac"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:1.37144em"><span style="top:-2.3139999999999996em"><span class="pstrut" style="height:3em"></span><span class="mord"><span class="mord" style="margin-right:.05556em">∂</span><span class="mord"><span class="mord mathnormal" style="margin-right:.02778em">θ</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:.31166399999999994em"><span style="top:-2.5500000000000003em;margin-left:-.02778em;margin-right:.05em"><span class="pstrut" style="height:2.7em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight">i</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.15em"><span></span></span></span></span></span></span></span></span><span style="top:-3.23em"><span class="pstrut" style="height:3em"></span><span class="frac-line" style="border-bottom-width:.04em"></span></span><span style="top:-3.677em"><span class="pstrut" style="height:3em"></span><span class="mord"><span class="mord" style="margin-right:.05556em">∂</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.8360000000000001em"><span></span></span></span></span></span><span class="mclose nulldelimiter"></span></span><span class="mord mathnormal" style="margin-right:.09618em">J</span><span class="mopen">(</span><span class="mord mathnormal" style="margin-right:.02778em">θ</span><span class="mclose">)</span></span></span></span></span></p><p>其中<span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>α</mi></mrow><annotation encoding="application/x-tex">α</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:.43056em;vertical-align:0"></span><span class="mord mathnormal" style="margin-right:.0037em">α</span></span></span></span> 为<strong>步长</strong>，也叫学习率。步长（Learning rate）决定了在梯度下降迭代的过程中，每一步沿梯度负方向前进的长度。用上面下山的例子，步长就是在当前这一步所在位置沿着最陡峭最易下山的位置走的那一步的长度。</p><p>步长合适：</p><p><img data-src="https://shun309.oss-cn-hangzhou.aliyuncs.com/photos/v2-aa03253883db6239bbdaa9db3ee6fcf5_hd.webp" alt=""></p><p>步长过小：</p><p><img data-src="https://shun309.oss-cn-hangzhou.aliyuncs.com/photos/v2-3c9538a23d6f453a698120c124bfa76e_hd.webp" alt=""></p><p>步长较大：</p><p><img data-src="https://shun309.oss-cn-hangzhou.aliyuncs.com/photos/v2-be1591ca247ab8485e13bb9725cb6f32_hd.webp" alt=""></p><p>步长过大：</p><p><img data-src="https://shun309.oss-cn-hangzhou.aliyuncs.com/photos/v2-5602652d76fc62b94fc7424ae87ba955_hd.webp" alt=""></p><p>总结：</p><p><img data-src="https://shun309.oss-cn-hangzhou.aliyuncs.com/photos/20210415215318.png" alt=""></p><h2 id="23-梯度下降法的一般步骤"><a class="anchor" href="#23-梯度下降法的一般步骤">#</a> 2.3 梯度下降法的一般步骤</h2><p>假设函数 <span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>y</mi><mo>=</mo><mi>f</mi><mo stretchy="false">(</mo><msub><mi>x</mi><mn>1</mn></msub><mo separator="true">,</mo><msub><mi>x</mi><mn>2</mn></msub><mo separator="true">,</mo><mi mathvariant="normal">.</mi><mi mathvariant="normal">.</mi><mi mathvariant="normal">.</mi><mo separator="true">,</mo><msub><mi>x</mi><mi>n</mi></msub><mo stretchy="false">)</mo></mrow><annotation encoding="application/x-tex">y = f (x_1,x_2,...,x_n)</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:.625em;vertical-align:-.19444em"></span><span class="mord mathnormal" style="margin-right:.03588em">y</span><span class="mspace" style="margin-right:.2777777777777778em"></span><span class="mrel">=</span><span class="mspace" style="margin-right:.2777777777777778em"></span></span><span class="base"><span class="strut" style="height:1em;vertical-align:-.25em"></span><span class="mord mathnormal" style="margin-right:.10764em">f</span><span class="mopen">(</span><span class="mord"><span class="mord mathnormal">x</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:.30110799999999993em"><span style="top:-2.5500000000000003em;margin-left:0;margin-right:.05em"><span class="pstrut" style="height:2.7em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">1</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.15em"><span></span></span></span></span></span></span><span class="mpunct">,</span><span class="mspace" style="margin-right:.16666666666666666em"></span><span class="mord"><span class="mord mathnormal">x</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:.30110799999999993em"><span style="top:-2.5500000000000003em;margin-left:0;margin-right:.05em"><span class="pstrut" style="height:2.7em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">2</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.15em"><span></span></span></span></span></span></span><span class="mpunct">,</span><span class="mspace" style="margin-right:.16666666666666666em"></span><span class="mord">.</span><span class="mord">.</span><span class="mord">.</span><span class="mpunct">,</span><span class="mspace" style="margin-right:.16666666666666666em"></span><span class="mord"><span class="mord mathnormal">x</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:.151392em"><span style="top:-2.5500000000000003em;margin-left:0;margin-right:.05em"><span class="pstrut" style="height:2.7em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight">n</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.15em"><span></span></span></span></span></span></span><span class="mclose">)</span></span></span></span> 只有一个极小点。<br>初始给定参数为 <span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><msub><mi>X</mi><mn>0</mn></msub><mo>=</mo><mo stretchy="false">(</mo><msub><mi>x</mi><mn>1</mn></msub><mn>0</mn><mo separator="true">,</mo><msub><mi>x</mi><mn>2</mn></msub><mn>0</mn><mo separator="true">,</mo><mi mathvariant="normal">.</mi><mi mathvariant="normal">.</mi><mi mathvariant="normal">.</mi><mo separator="true">,</mo><msub><mi>x</mi><mi>n</mi></msub><mn>0</mn><mo separator="true">,</mo><mo stretchy="false">)</mo></mrow><annotation encoding="application/x-tex">X_0 = (x_10,x_20,...,x_n0,)</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:.83333em;vertical-align:-.15em"></span><span class="mord"><span class="mord mathnormal" style="margin-right:.07847em">X</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:.30110799999999993em"><span style="top:-2.5500000000000003em;margin-left:-.07847em;margin-right:.05em"><span class="pstrut" style="height:2.7em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">0</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.15em"><span></span></span></span></span></span></span><span class="mspace" style="margin-right:.2777777777777778em"></span><span class="mrel">=</span><span class="mspace" style="margin-right:.2777777777777778em"></span></span><span class="base"><span class="strut" style="height:1em;vertical-align:-.25em"></span><span class="mopen">(</span><span class="mord"><span class="mord mathnormal">x</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:.30110799999999993em"><span style="top:-2.5500000000000003em;margin-left:0;margin-right:.05em"><span class="pstrut" style="height:2.7em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">1</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.15em"><span></span></span></span></span></span></span><span class="mord">0</span><span class="mpunct">,</span><span class="mspace" style="margin-right:.16666666666666666em"></span><span class="mord"><span class="mord mathnormal">x</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:.30110799999999993em"><span style="top:-2.5500000000000003em;margin-left:0;margin-right:.05em"><span class="pstrut" style="height:2.7em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">2</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.15em"><span></span></span></span></span></span></span><span class="mord">0</span><span class="mpunct">,</span><span class="mspace" style="margin-right:.16666666666666666em"></span><span class="mord">.</span><span class="mord">.</span><span class="mord">.</span><span class="mpunct">,</span><span class="mspace" style="margin-right:.16666666666666666em"></span><span class="mord"><span class="mord mathnormal">x</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:.151392em"><span style="top:-2.5500000000000003em;margin-left:0;margin-right:.05em"><span class="pstrut" style="height:2.7em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight">n</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.15em"><span></span></span></span></span></span></span><span class="mord">0</span><span class="mpunct">,</span><span class="mspace" style="margin-right:.16666666666666666em"></span><span class="mclose">)</span></span></span></span>。从这个点如何搜索才能找到原函数的极小值点？<br>方法：<br>①首先设定一个较小的正数<span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>α</mi></mrow><annotation encoding="application/x-tex">α</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:.43056em;vertical-align:0"></span><span class="mord mathnormal" style="margin-right:.0037em">α</span></span></span></span>，<span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>ε</mi></mrow><annotation encoding="application/x-tex">ε</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:.43056em;vertical-align:0"></span><span class="mord mathnormal">ε</span></span></span></span>；<br>②求当前位置出处的各个偏导数：</p><p><span class="katex-display"><span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML" display="block"><semantics><mrow><msup><mi>f</mi><mo mathvariant="normal" lspace="0em" rspace="0em">′</mo></msup><mo stretchy="false">(</mo><msub><mi>x</mi><mrow><mi>m</mi><mn>0</mn></mrow></msub><mo stretchy="false">)</mo><mo>=</mo><mfrac><mrow><mi mathvariant="normal">∂</mi><mi>y</mi></mrow><mrow><mi mathvariant="normal">∂</mi><msub><mi>x</mi><mi>m</mi></msub></mrow></mfrac><mo stretchy="false">(</mo><msub><mi>x</mi><mrow><mi>m</mi><mn>0</mn></mrow></msub><mo stretchy="false">)</mo><mo separator="true">,</mo><mi>m</mi><mo>=</mo><mn>1</mn><mo separator="true">,</mo><mn>2</mn><mo separator="true">,</mo><mi mathvariant="normal">.</mi><mi mathvariant="normal">.</mi><mi mathvariant="normal">.</mi><mo separator="true">,</mo><mi>n</mi></mrow><annotation encoding="application/x-tex">f&#x27;(x_{m0})=\frac{∂y}{∂x_{m}}(x_{m0}),m=1,2,...,n</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:1.051892em;vertical-align:-.25em"></span><span class="mord"><span class="mord mathnormal" style="margin-right:.10764em">f</span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height:.801892em"><span style="top:-3.113em;margin-right:.05em"><span class="pstrut" style="height:2.7em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mtight">′</span></span></span></span></span></span></span></span></span><span class="mopen">(</span><span class="mord"><span class="mord mathnormal">x</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:.30110799999999993em"><span style="top:-2.5500000000000003em;margin-left:0;margin-right:.05em"><span class="pstrut" style="height:2.7em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight">m</span><span class="mord mtight">0</span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.15em"><span></span></span></span></span></span></span><span class="mclose">)</span><span class="mspace" style="margin-right:.2777777777777778em"></span><span class="mrel">=</span><span class="mspace" style="margin-right:.2777777777777778em"></span></span><span class="base"><span class="strut" style="height:2.20744em;vertical-align:-.8360000000000001em"></span><span class="mord"><span class="mopen nulldelimiter"></span><span class="mfrac"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:1.3714399999999998em"><span style="top:-2.3139999999999996em"><span class="pstrut" style="height:3em"></span><span class="mord"><span class="mord" style="margin-right:.05556em">∂</span><span class="mord"><span class="mord mathnormal">x</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:.151392em"><span style="top:-2.5500000000000003em;margin-left:0;margin-right:.05em"><span class="pstrut" style="height:2.7em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight">m</span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.15em"><span></span></span></span></span></span></span></span></span><span style="top:-3.23em"><span class="pstrut" style="height:3em"></span><span class="frac-line" style="border-bottom-width:.04em"></span></span><span style="top:-3.677em"><span class="pstrut" style="height:3em"></span><span class="mord"><span class="mord" style="margin-right:.05556em">∂</span><span class="mord mathnormal" style="margin-right:.03588em">y</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.8360000000000001em"><span></span></span></span></span></span><span class="mclose nulldelimiter"></span></span><span class="mopen">(</span><span class="mord"><span class="mord mathnormal">x</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:.30110799999999993em"><span style="top:-2.5500000000000003em;margin-left:0;margin-right:.05em"><span class="pstrut" style="height:2.7em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight">m</span><span class="mord mtight">0</span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.15em"><span></span></span></span></span></span></span><span class="mclose">)</span><span class="mpunct">,</span><span class="mspace" style="margin-right:.16666666666666666em"></span><span class="mord mathnormal">m</span><span class="mspace" style="margin-right:.2777777777777778em"></span><span class="mrel">=</span><span class="mspace" style="margin-right:.2777777777777778em"></span></span><span class="base"><span class="strut" style="height:.8388800000000001em;vertical-align:-.19444em"></span><span class="mord">1</span><span class="mpunct">,</span><span class="mspace" style="margin-right:.16666666666666666em"></span><span class="mord">2</span><span class="mpunct">,</span><span class="mspace" style="margin-right:.16666666666666666em"></span><span class="mord">.</span><span class="mord">.</span><span class="mord">.</span><span class="mpunct">,</span><span class="mspace" style="margin-right:.16666666666666666em"></span><span class="mord mathnormal">n</span></span></span></span></span></p><p>③修改当前函数的参数值，公式如下：</p><p><span class="katex-display"><span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML" display="block"><semantics><mrow><msubsup><mi>x</mi><mi>m</mi><mo mathvariant="normal" lspace="0em" rspace="0em">′</mo></msubsup><mo>=</mo><msub><mi>x</mi><mi>m</mi></msub><mo>−</mo><mi>α</mi><mfrac><mrow><mi mathvariant="normal">∂</mi><mi>y</mi></mrow><mrow><mi mathvariant="normal">∂</mi><msub><mi>x</mi><mi>m</mi></msub></mrow></mfrac><mo stretchy="false">(</mo><msub><mi>x</mi><mrow><mi>m</mi><mn>0</mn></mrow></msub><mo stretchy="false">)</mo><mo separator="true">,</mo><mi>m</mi><mo>=</mo><mn>1</mn><mo separator="true">,</mo><mn>2</mn><mo separator="true">,</mo><mi mathvariant="normal">.</mi><mi mathvariant="normal">.</mi><mi mathvariant="normal">.</mi><mo separator="true">,</mo><mi>n</mi></mrow><annotation encoding="application/x-tex">x&#x27;_{m}=x_{m}-α\frac{∂y}{∂x_{m}}(x_{m0}),m=1,2,...,n</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:1.048892em;vertical-align:-.247em"></span><span class="mord"><span class="mord mathnormal">x</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:.8018919999999999em"><span style="top:-2.4530000000000003em;margin-left:0;margin-right:.05em"><span class="pstrut" style="height:2.7em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight">m</span></span></span></span><span style="top:-3.113em;margin-right:.05em"><span class="pstrut" style="height:2.7em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mtight">′</span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.247em"><span></span></span></span></span></span></span><span class="mspace" style="margin-right:.2777777777777778em"></span><span class="mrel">=</span><span class="mspace" style="margin-right:.2777777777777778em"></span></span><span class="base"><span class="strut" style="height:.73333em;vertical-align:-.15em"></span><span class="mord"><span class="mord mathnormal">x</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:.151392em"><span style="top:-2.5500000000000003em;margin-left:0;margin-right:.05em"><span class="pstrut" style="height:2.7em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight">m</span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.15em"><span></span></span></span></span></span></span><span class="mspace" style="margin-right:.2222222222222222em"></span><span class="mbin">−</span><span class="mspace" style="margin-right:.2222222222222222em"></span></span><span class="base"><span class="strut" style="height:2.20744em;vertical-align:-.8360000000000001em"></span><span class="mord mathnormal" style="margin-right:.0037em">α</span><span class="mord"><span class="mopen nulldelimiter"></span><span class="mfrac"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:1.3714399999999998em"><span style="top:-2.3139999999999996em"><span class="pstrut" style="height:3em"></span><span class="mord"><span class="mord" style="margin-right:.05556em">∂</span><span class="mord"><span class="mord mathnormal">x</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:.151392em"><span style="top:-2.5500000000000003em;margin-left:0;margin-right:.05em"><span class="pstrut" style="height:2.7em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight">m</span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.15em"><span></span></span></span></span></span></span></span></span><span style="top:-3.23em"><span class="pstrut" style="height:3em"></span><span class="frac-line" style="border-bottom-width:.04em"></span></span><span style="top:-3.677em"><span class="pstrut" style="height:3em"></span><span class="mord"><span class="mord" style="margin-right:.05556em">∂</span><span class="mord mathnormal" style="margin-right:.03588em">y</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.8360000000000001em"><span></span></span></span></span></span><span class="mclose nulldelimiter"></span></span><span class="mopen">(</span><span class="mord"><span class="mord mathnormal">x</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:.30110799999999993em"><span style="top:-2.5500000000000003em;margin-left:0;margin-right:.05em"><span class="pstrut" style="height:2.7em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight">m</span><span class="mord mtight">0</span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.15em"><span></span></span></span></span></span></span><span class="mclose">)</span><span class="mpunct">,</span><span class="mspace" style="margin-right:.16666666666666666em"></span><span class="mord mathnormal">m</span><span class="mspace" style="margin-right:.2777777777777778em"></span><span class="mrel">=</span><span class="mspace" style="margin-right:.2777777777777778em"></span></span><span class="base"><span class="strut" style="height:.8388800000000001em;vertical-align:-.19444em"></span><span class="mord">1</span><span class="mpunct">,</span><span class="mspace" style="margin-right:.16666666666666666em"></span><span class="mord">2</span><span class="mpunct">,</span><span class="mspace" style="margin-right:.16666666666666666em"></span><span class="mord">.</span><span class="mord">.</span><span class="mord">.</span><span class="mpunct">,</span><span class="mspace" style="margin-right:.16666666666666666em"></span><span class="mord mathnormal">n</span></span></span></span></span></p><p>④如果参数变化量小于<span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>ε</mi></mrow><annotation encoding="application/x-tex">ε</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:.43056em;vertical-align:0"></span><span class="mord mathnormal">ε</span></span></span></span>，退出；否则返回第 2 步。</p><h2 id="24-一元线性回归函数推导过程"><a class="anchor" href="#24-一元线性回归函数推导过程">#</a> 2.4 一元线性回归函数推导过程</h2><p>设线性回归函数：<span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mover accent="true"><mi>y</mi><mo>^</mo></mover><mo>=</mo><mi>w</mi><mi>x</mi><mo>+</mo><mi>b</mi></mrow><annotation encoding="application/x-tex">\hat{y}=wx+b</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:.8888799999999999em;vertical-align:-.19444em"></span><span class="mord accent"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:.69444em"><span style="top:-3em"><span class="pstrut" style="height:3em"></span><span class="mord"><span class="mord mathnormal" style="margin-right:.03588em">y</span></span></span><span style="top:-3em"><span class="pstrut" style="height:3em"></span><span class="accent-body" style="left:-.19444em"><span class="mord">^</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.19444em"><span></span></span></span></span></span><span class="mspace" style="margin-right:.2777777777777778em"></span><span class="mrel">=</span><span class="mspace" style="margin-right:.2777777777777778em"></span></span><span class="base"><span class="strut" style="height:.66666em;vertical-align:-.08333em"></span><span class="mord mathnormal" style="margin-right:.02691em">w</span><span class="mord mathnormal">x</span><span class="mspace" style="margin-right:.2222222222222222em"></span><span class="mbin">+</span><span class="mspace" style="margin-right:.2222222222222222em"></span></span><span class="base"><span class="strut" style="height:.69444em;vertical-align:0"></span><span class="mord mathnormal">b</span></span></span></span><br>构造损失函数（loss）：</p><p><span class="katex-display"><span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML" display="block"><semantics><mrow><mi>L</mi><mo stretchy="false">(</mo><mi>w</mi><mo separator="true">,</mo><mi>b</mi><mo stretchy="false">)</mo><mo>=</mo><mfrac><mn>1</mn><mrow><mn>2</mn><mi>n</mi></mrow></mfrac><munderover><mo>∑</mo><mrow><mi>i</mi><mo>=</mo><mn>1</mn></mrow><mi>n</mi></munderover><mo stretchy="false">(</mo><mi>w</mi><msub><mi>x</mi><mi>i</mi></msub><mo>+</mo><mi>b</mi><mo>−</mo><msub><mi>y</mi><mi>i</mi></msub><msup><mo stretchy="false">)</mo><mn>2</mn></msup></mrow><annotation encoding="application/x-tex">L(w,b)=\frac{1}{2n}\sum^n_{i=1}(wx_i+b-y_i)^2</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:1em;vertical-align:-.25em"></span><span class="mord mathnormal">L</span><span class="mopen">(</span><span class="mord mathnormal" style="margin-right:.02691em">w</span><span class="mpunct">,</span><span class="mspace" style="margin-right:.16666666666666666em"></span><span class="mord mathnormal">b</span><span class="mclose">)</span><span class="mspace" style="margin-right:.2777777777777778em"></span><span class="mrel">=</span><span class="mspace" style="margin-right:.2777777777777778em"></span></span><span class="base"><span class="strut" style="height:2.929066em;vertical-align:-1.277669em"></span><span class="mord"><span class="mopen nulldelimiter"></span><span class="mfrac"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:1.32144em"><span style="top:-2.314em"><span class="pstrut" style="height:3em"></span><span class="mord"><span class="mord">2</span><span class="mord mathnormal">n</span></span></span><span style="top:-3.23em"><span class="pstrut" style="height:3em"></span><span class="frac-line" style="border-bottom-width:.04em"></span></span><span style="top:-3.677em"><span class="pstrut" style="height:3em"></span><span class="mord"><span class="mord">1</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.686em"><span></span></span></span></span></span><span class="mclose nulldelimiter"></span></span><span class="mspace" style="margin-right:.16666666666666666em"></span><span class="mop op-limits"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:1.6513970000000002em"><span style="top:-1.872331em;margin-left:0"><span class="pstrut" style="height:3.05em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight">i</span><span class="mrel mtight">=</span><span class="mord mtight">1</span></span></span></span><span style="top:-3.050005em"><span class="pstrut" style="height:3.05em"></span><span><span class="mop op-symbol large-op">∑</span></span></span><span style="top:-4.3000050000000005em;margin-left:0"><span class="pstrut" style="height:3.05em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight">n</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:1.277669em"><span></span></span></span></span></span><span class="mopen">(</span><span class="mord mathnormal" style="margin-right:.02691em">w</span><span class="mord"><span class="mord mathnormal">x</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:.31166399999999994em"><span style="top:-2.5500000000000003em;margin-left:0;margin-right:.05em"><span class="pstrut" style="height:2.7em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight">i</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.15em"><span></span></span></span></span></span></span><span class="mspace" style="margin-right:.2222222222222222em"></span><span class="mbin">+</span><span class="mspace" style="margin-right:.2222222222222222em"></span></span><span class="base"><span class="strut" style="height:.77777em;vertical-align:-.08333em"></span><span class="mord mathnormal">b</span><span class="mspace" style="margin-right:.2222222222222222em"></span><span class="mbin">−</span><span class="mspace" style="margin-right:.2222222222222222em"></span></span><span class="base"><span class="strut" style="height:1.1141079999999999em;vertical-align:-.25em"></span><span class="mord"><span class="mord mathnormal" style="margin-right:.03588em">y</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:.31166399999999994em"><span style="top:-2.5500000000000003em;margin-left:-.03588em;margin-right:.05em"><span class="pstrut" style="height:2.7em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight">i</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.15em"><span></span></span></span></span></span></span><span class="mclose"><span class="mclose">)</span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height:.8641079999999999em"><span style="top:-3.113em;margin-right:.05em"><span class="pstrut" style="height:2.7em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">2</span></span></span></span></span></span></span></span></span></span></span></span></p><p>思路：通过梯度下降法不断更新<span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>w</mi></mrow><annotation encoding="application/x-tex">w</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:.43056em;vertical-align:0"></span><span class="mord mathnormal" style="margin-right:.02691em">w</span></span></span></span> 和<span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>b</mi></mrow><annotation encoding="application/x-tex">b</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:.69444em;vertical-align:0"></span><span class="mord mathnormal">b</span></span></span></span>，当损失函数的值特别小时，就得到了我们最终的函数模型。<br>过程：</p><p>step1. 求导：</p><p><span class="katex-display"><span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML" display="block"><semantics><mrow><mfrac><mi mathvariant="normal">∂</mi><mrow><mi mathvariant="normal">∂</mi><mi>w</mi></mrow></mfrac><mi>L</mi><mo stretchy="false">(</mo><mi>w</mi><mo separator="true">,</mo><mi>b</mi><mo stretchy="false">)</mo><mo>=</mo><mfrac><mn>1</mn><mi>n</mi></mfrac><munderover><mo>∑</mo><mrow><mi>i</mi><mo>=</mo><mn>1</mn></mrow><mi>n</mi></munderover><mo stretchy="false">(</mo><mo stretchy="false">(</mo><mi>w</mi><msub><mi>x</mi><mi>i</mi></msub><mo>+</mo><mi>b</mi><mo>−</mo><msub><mi>y</mi><mi>i</mi></msub><mo stretchy="false">)</mo><mo separator="true">⋅</mo><msub><mi>x</mi><mi>i</mi></msub><mo stretchy="false">)</mo></mrow><annotation encoding="application/x-tex">\frac{∂}{∂w}L(w,b)=\frac{1}{n}\sum^n_{i=1}((wx_i+b-y_i)·x_i)</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:2.05744em;vertical-align:-.686em"></span><span class="mord"><span class="mopen nulldelimiter"></span><span class="mfrac"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:1.37144em"><span style="top:-2.314em"><span class="pstrut" style="height:3em"></span><span class="mord"><span class="mord" style="margin-right:.05556em">∂</span><span class="mord mathnormal" style="margin-right:.02691em">w</span></span></span><span style="top:-3.23em"><span class="pstrut" style="height:3em"></span><span class="frac-line" style="border-bottom-width:.04em"></span></span><span style="top:-3.677em"><span class="pstrut" style="height:3em"></span><span class="mord"><span class="mord" style="margin-right:.05556em">∂</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.686em"><span></span></span></span></span></span><span class="mclose nulldelimiter"></span></span><span class="mord mathnormal">L</span><span class="mopen">(</span><span class="mord mathnormal" style="margin-right:.02691em">w</span><span class="mpunct">,</span><span class="mspace" style="margin-right:.16666666666666666em"></span><span class="mord mathnormal">b</span><span class="mclose">)</span><span class="mspace" style="margin-right:.2777777777777778em"></span><span class="mrel">=</span><span class="mspace" style="margin-right:.2777777777777778em"></span></span><span class="base"><span class="strut" style="height:2.929066em;vertical-align:-1.277669em"></span><span class="mord"><span class="mopen nulldelimiter"></span><span class="mfrac"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:1.32144em"><span style="top:-2.314em"><span class="pstrut" style="height:3em"></span><span class="mord"><span class="mord mathnormal">n</span></span></span><span style="top:-3.23em"><span class="pstrut" style="height:3em"></span><span class="frac-line" style="border-bottom-width:.04em"></span></span><span style="top:-3.677em"><span class="pstrut" style="height:3em"></span><span class="mord"><span class="mord">1</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.686em"><span></span></span></span></span></span><span class="mclose nulldelimiter"></span></span><span class="mspace" style="margin-right:.16666666666666666em"></span><span class="mop op-limits"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:1.6513970000000002em"><span style="top:-1.872331em;margin-left:0"><span class="pstrut" style="height:3.05em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight">i</span><span class="mrel mtight">=</span><span class="mord mtight">1</span></span></span></span><span style="top:-3.050005em"><span class="pstrut" style="height:3.05em"></span><span><span class="mop op-symbol large-op">∑</span></span></span><span style="top:-4.3000050000000005em;margin-left:0"><span class="pstrut" style="height:3.05em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight">n</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:1.277669em"><span></span></span></span></span></span><span class="mopen">(</span><span class="mopen">(</span><span class="mord mathnormal" style="margin-right:.02691em">w</span><span class="mord"><span class="mord mathnormal">x</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:.31166399999999994em"><span style="top:-2.5500000000000003em;margin-left:0;margin-right:.05em"><span class="pstrut" style="height:2.7em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight">i</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.15em"><span></span></span></span></span></span></span><span class="mspace" style="margin-right:.2222222222222222em"></span><span class="mbin">+</span><span class="mspace" style="margin-right:.2222222222222222em"></span></span><span class="base"><span class="strut" style="height:.77777em;vertical-align:-.08333em"></span><span class="mord mathnormal">b</span><span class="mspace" style="margin-right:.2222222222222222em"></span><span class="mbin">−</span><span class="mspace" style="margin-right:.2222222222222222em"></span></span><span class="base"><span class="strut" style="height:1em;vertical-align:-.25em"></span><span class="mord"><span class="mord mathnormal" style="margin-right:.03588em">y</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:.31166399999999994em"><span style="top:-2.5500000000000003em;margin-left:-.03588em;margin-right:.05em"><span class="pstrut" style="height:2.7em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight">i</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.15em"><span></span></span></span></span></span></span><span class="mclose">)</span><span class="mpunct">⋅</span><span class="mspace" style="margin-right:.16666666666666666em"></span><span class="mord"><span class="mord mathnormal">x</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:.31166399999999994em"><span style="top:-2.5500000000000003em;margin-left:0;margin-right:.05em"><span class="pstrut" style="height:2.7em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight">i</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.15em"><span></span></span></span></span></span></span><span class="mclose">)</span></span></span></span></span></p><p><span class="katex-display"><span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML" display="block"><semantics><mrow><mfrac><mi mathvariant="normal">∂</mi><mrow><mi mathvariant="normal">∂</mi><mi>b</mi></mrow></mfrac><mi>L</mi><mo stretchy="false">(</mo><mi>w</mi><mo separator="true">,</mo><mi>b</mi><mo stretchy="false">)</mo><mo>=</mo><mfrac><mn>1</mn><mi>n</mi></mfrac><munderover><mo>∑</mo><mrow><mi>i</mi><mo>=</mo><mn>1</mn></mrow><mi>n</mi></munderover><mo stretchy="false">(</mo><mi>w</mi><msub><mi>x</mi><mi>i</mi></msub><mo>+</mo><mi>b</mi><mo>−</mo><msub><mi>y</mi><mi>i</mi></msub><mo stretchy="false">)</mo></mrow><annotation encoding="application/x-tex">\frac{∂}{∂b}L(w,b)=\frac{1}{n}\sum^n_{i=1}(wx_i+b-y_i)</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:2.05744em;vertical-align:-.686em"></span><span class="mord"><span class="mopen nulldelimiter"></span><span class="mfrac"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:1.37144em"><span style="top:-2.314em"><span class="pstrut" style="height:3em"></span><span class="mord"><span class="mord" style="margin-right:.05556em">∂</span><span class="mord mathnormal">b</span></span></span><span style="top:-3.23em"><span class="pstrut" style="height:3em"></span><span class="frac-line" style="border-bottom-width:.04em"></span></span><span style="top:-3.677em"><span class="pstrut" style="height:3em"></span><span class="mord"><span class="mord" style="margin-right:.05556em">∂</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.686em"><span></span></span></span></span></span><span class="mclose nulldelimiter"></span></span><span class="mord mathnormal">L</span><span class="mopen">(</span><span class="mord mathnormal" style="margin-right:.02691em">w</span><span class="mpunct">,</span><span class="mspace" style="margin-right:.16666666666666666em"></span><span class="mord mathnormal">b</span><span class="mclose">)</span><span class="mspace" style="margin-right:.2777777777777778em"></span><span class="mrel">=</span><span class="mspace" style="margin-right:.2777777777777778em"></span></span><span class="base"><span class="strut" style="height:2.929066em;vertical-align:-1.277669em"></span><span class="mord"><span class="mopen nulldelimiter"></span><span class="mfrac"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:1.32144em"><span style="top:-2.314em"><span class="pstrut" style="height:3em"></span><span class="mord"><span class="mord mathnormal">n</span></span></span><span style="top:-3.23em"><span class="pstrut" style="height:3em"></span><span class="frac-line" style="border-bottom-width:.04em"></span></span><span style="top:-3.677em"><span class="pstrut" style="height:3em"></span><span class="mord"><span class="mord">1</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.686em"><span></span></span></span></span></span><span class="mclose nulldelimiter"></span></span><span class="mspace" style="margin-right:.16666666666666666em"></span><span class="mop op-limits"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:1.6513970000000002em"><span style="top:-1.872331em;margin-left:0"><span class="pstrut" style="height:3.05em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight">i</span><span class="mrel mtight">=</span><span class="mord mtight">1</span></span></span></span><span style="top:-3.050005em"><span class="pstrut" style="height:3.05em"></span><span><span class="mop op-symbol large-op">∑</span></span></span><span style="top:-4.3000050000000005em;margin-left:0"><span class="pstrut" style="height:3.05em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight">n</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:1.277669em"><span></span></span></span></span></span><span class="mopen">(</span><span class="mord mathnormal" style="margin-right:.02691em">w</span><span class="mord"><span class="mord mathnormal">x</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:.31166399999999994em"><span style="top:-2.5500000000000003em;margin-left:0;margin-right:.05em"><span class="pstrut" style="height:2.7em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight">i</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.15em"><span></span></span></span></span></span></span><span class="mspace" style="margin-right:.2222222222222222em"></span><span class="mbin">+</span><span class="mspace" style="margin-right:.2222222222222222em"></span></span><span class="base"><span class="strut" style="height:.77777em;vertical-align:-.08333em"></span><span class="mord mathnormal">b</span><span class="mspace" style="margin-right:.2222222222222222em"></span><span class="mbin">−</span><span class="mspace" style="margin-right:.2222222222222222em"></span></span><span class="base"><span class="strut" style="height:1em;vertical-align:-.25em"></span><span class="mord"><span class="mord mathnormal" style="margin-right:.03588em">y</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:.31166399999999994em"><span style="top:-2.5500000000000003em;margin-left:-.03588em;margin-right:.05em"><span class="pstrut" style="height:2.7em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight">i</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.15em"><span></span></span></span></span></span></span><span class="mclose">)</span></span></span></span></span></p><p>step2. 更新<span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><msub><mi>θ</mi><mn>0</mn></msub></mrow><annotation encoding="application/x-tex">θ_{0}</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:.84444em;vertical-align:-.15em"></span><span class="mord"><span class="mord mathnormal" style="margin-right:.02778em">θ</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:.30110799999999993em"><span style="top:-2.5500000000000003em;margin-left:-.02778em;margin-right:.05em"><span class="pstrut" style="height:2.7em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mtight">0</span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.15em"><span></span></span></span></span></span></span></span></span></span> 和<span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><msub><mi>θ</mi><mn>1</mn></msub></mrow><annotation encoding="application/x-tex">θ_{1}</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:.84444em;vertical-align:-.15em"></span><span class="mord"><span class="mord mathnormal" style="margin-right:.02778em">θ</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:.30110799999999993em"><span style="top:-2.5500000000000003em;margin-left:-.02778em;margin-right:.05em"><span class="pstrut" style="height:2.7em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mtight">1</span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.15em"><span></span></span></span></span></span></span></span></span></span>：</p><p><span class="katex-display"><span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML" display="block"><semantics><mrow><mi>w</mi><mo>=</mo><mi>w</mi><mo>−</mo><mi>α</mi><mo separator="true">⋅</mo><mfrac><mn>1</mn><mi>n</mi></mfrac><munderover><mo>∑</mo><mrow><mi>i</mi><mo>=</mo><mn>1</mn></mrow><mi>n</mi></munderover><mo stretchy="false">(</mo><mo stretchy="false">(</mo><mi>w</mi><msub><mi>x</mi><mi>i</mi></msub><mo>+</mo><mi>b</mi><mo>−</mo><msub><mi>y</mi><mi>i</mi></msub><mo stretchy="false">)</mo><mo separator="true">⋅</mo><msub><mi>x</mi><mi>i</mi></msub><mo stretchy="false">)</mo></mrow><annotation encoding="application/x-tex">w=w-α·\frac{1}{n}\sum^n_{i=1}((wx_i+b-y_i)·x_i)</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:.43056em;vertical-align:0"></span><span class="mord mathnormal" style="margin-right:.02691em">w</span><span class="mspace" style="margin-right:.2777777777777778em"></span><span class="mrel">=</span><span class="mspace" style="margin-right:.2777777777777778em"></span></span><span class="base"><span class="strut" style="height:.66666em;vertical-align:-.08333em"></span><span class="mord mathnormal" style="margin-right:.02691em">w</span><span class="mspace" style="margin-right:.2222222222222222em"></span><span class="mbin">−</span><span class="mspace" style="margin-right:.2222222222222222em"></span></span><span class="base"><span class="strut" style="height:2.929066em;vertical-align:-1.277669em"></span><span class="mord mathnormal" style="margin-right:.0037em">α</span><span class="mpunct">⋅</span><span class="mspace" style="margin-right:.16666666666666666em"></span><span class="mord"><span class="mopen nulldelimiter"></span><span class="mfrac"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:1.32144em"><span style="top:-2.314em"><span class="pstrut" style="height:3em"></span><span class="mord"><span class="mord mathnormal">n</span></span></span><span style="top:-3.23em"><span class="pstrut" style="height:3em"></span><span class="frac-line" style="border-bottom-width:.04em"></span></span><span style="top:-3.677em"><span class="pstrut" style="height:3em"></span><span class="mord"><span class="mord">1</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.686em"><span></span></span></span></span></span><span class="mclose nulldelimiter"></span></span><span class="mspace" style="margin-right:.16666666666666666em"></span><span class="mop op-limits"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:1.6513970000000002em"><span style="top:-1.872331em;margin-left:0"><span class="pstrut" style="height:3.05em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight">i</span><span class="mrel mtight">=</span><span class="mord mtight">1</span></span></span></span><span style="top:-3.050005em"><span class="pstrut" style="height:3.05em"></span><span><span class="mop op-symbol large-op">∑</span></span></span><span style="top:-4.3000050000000005em;margin-left:0"><span class="pstrut" style="height:3.05em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight">n</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:1.277669em"><span></span></span></span></span></span><span class="mopen">(</span><span class="mopen">(</span><span class="mord mathnormal" style="margin-right:.02691em">w</span><span class="mord"><span class="mord mathnormal">x</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:.31166399999999994em"><span style="top:-2.5500000000000003em;margin-left:0;margin-right:.05em"><span class="pstrut" style="height:2.7em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight">i</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.15em"><span></span></span></span></span></span></span><span class="mspace" style="margin-right:.2222222222222222em"></span><span class="mbin">+</span><span class="mspace" style="margin-right:.2222222222222222em"></span></span><span class="base"><span class="strut" style="height:.77777em;vertical-align:-.08333em"></span><span class="mord mathnormal">b</span><span class="mspace" style="margin-right:.2222222222222222em"></span><span class="mbin">−</span><span class="mspace" style="margin-right:.2222222222222222em"></span></span><span class="base"><span class="strut" style="height:1em;vertical-align:-.25em"></span><span class="mord"><span class="mord mathnormal" style="margin-right:.03588em">y</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:.31166399999999994em"><span style="top:-2.5500000000000003em;margin-left:-.03588em;margin-right:.05em"><span class="pstrut" style="height:2.7em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight">i</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.15em"><span></span></span></span></span></span></span><span class="mclose">)</span><span class="mpunct">⋅</span><span class="mspace" style="margin-right:.16666666666666666em"></span><span class="mord"><span class="mord mathnormal">x</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:.31166399999999994em"><span style="top:-2.5500000000000003em;margin-left:0;margin-right:.05em"><span class="pstrut" style="height:2.7em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight">i</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.15em"><span></span></span></span></span></span></span><span class="mclose">)</span></span></span></span></span></p><p><span class="katex-display"><span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML" display="block"><semantics><mrow><mi>b</mi><mo>=</mo><mi>b</mi><mo>−</mo><mi>α</mi><mo separator="true">⋅</mo><mfrac><mn>1</mn><mi>n</mi></mfrac><munderover><mo>∑</mo><mrow><mi>i</mi><mo>=</mo><mn>1</mn></mrow><mi>n</mi></munderover><mo stretchy="false">(</mo><mi>w</mi><msub><mi>x</mi><mi>i</mi></msub><mo>+</mo><mi>b</mi><mo>−</mo><msub><mi>y</mi><mi>i</mi></msub><mo stretchy="false">)</mo></mrow><annotation encoding="application/x-tex">b=b-α·\frac{1}{n}\sum^n_{i=1}(wx_i+b-y_i)</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:.69444em;vertical-align:0"></span><span class="mord mathnormal">b</span><span class="mspace" style="margin-right:.2777777777777778em"></span><span class="mrel">=</span><span class="mspace" style="margin-right:.2777777777777778em"></span></span><span class="base"><span class="strut" style="height:.77777em;vertical-align:-.08333em"></span><span class="mord mathnormal">b</span><span class="mspace" style="margin-right:.2222222222222222em"></span><span class="mbin">−</span><span class="mspace" style="margin-right:.2222222222222222em"></span></span><span class="base"><span class="strut" style="height:2.929066em;vertical-align:-1.277669em"></span><span class="mord mathnormal" style="margin-right:.0037em">α</span><span class="mpunct">⋅</span><span class="mspace" style="margin-right:.16666666666666666em"></span><span class="mord"><span class="mopen nulldelimiter"></span><span class="mfrac"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:1.32144em"><span style="top:-2.314em"><span class="pstrut" style="height:3em"></span><span class="mord"><span class="mord mathnormal">n</span></span></span><span style="top:-3.23em"><span class="pstrut" style="height:3em"></span><span class="frac-line" style="border-bottom-width:.04em"></span></span><span style="top:-3.677em"><span class="pstrut" style="height:3em"></span><span class="mord"><span class="mord">1</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.686em"><span></span></span></span></span></span><span class="mclose nulldelimiter"></span></span><span class="mspace" style="margin-right:.16666666666666666em"></span><span class="mop op-limits"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:1.6513970000000002em"><span style="top:-1.872331em;margin-left:0"><span class="pstrut" style="height:3.05em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight">i</span><span class="mrel mtight">=</span><span class="mord mtight">1</span></span></span></span><span style="top:-3.050005em"><span class="pstrut" style="height:3.05em"></span><span><span class="mop op-symbol large-op">∑</span></span></span><span style="top:-4.3000050000000005em;margin-left:0"><span class="pstrut" style="height:3.05em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight">n</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:1.277669em"><span></span></span></span></span></span><span class="mopen">(</span><span class="mord mathnormal" style="margin-right:.02691em">w</span><span class="mord"><span class="mord mathnormal">x</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:.31166399999999994em"><span style="top:-2.5500000000000003em;margin-left:0;margin-right:.05em"><span class="pstrut" style="height:2.7em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight">i</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.15em"><span></span></span></span></span></span></span><span class="mspace" style="margin-right:.2222222222222222em"></span><span class="mbin">+</span><span class="mspace" style="margin-right:.2222222222222222em"></span></span><span class="base"><span class="strut" style="height:.77777em;vertical-align:-.08333em"></span><span class="mord mathnormal">b</span><span class="mspace" style="margin-right:.2222222222222222em"></span><span class="mbin">−</span><span class="mspace" style="margin-right:.2222222222222222em"></span></span><span class="base"><span class="strut" style="height:1em;vertical-align:-.25em"></span><span class="mord"><span class="mord mathnormal" style="margin-right:.03588em">y</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:.31166399999999994em"><span style="top:-2.5500000000000003em;margin-left:-.03588em;margin-right:.05em"><span class="pstrut" style="height:2.7em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight">i</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.15em"><span></span></span></span></span></span></span><span class="mclose">)</span></span></span></span></span></p><p>step3. 代入损失函数，求损失函数的值，若得到的值小于<span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>ε</mi></mrow><annotation encoding="application/x-tex">ε</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:.43056em;vertical-align:0"></span><span class="mord mathnormal">ε</span></span></span></span>（一般为 0.01 或者 0.001 这样的小数），退出；否则，返回 step1。</p><h2 id="25-梯度下降存在的问题"><a class="anchor" href="#25-梯度下降存在的问题">#</a> 2.5 梯度下降存在的问题：</h2><ul><li>当靠近极小值时收敛速度减慢；</li><li>直线搜索时可能会产生一些问题；</li><li>下降过程可能会出现 “之字形” 地下降。</li></ul><h2 id="26-梯度下降三兄弟bgdsgdmbgd"><a class="anchor" href="#26-梯度下降三兄弟bgdsgdmbgd">#</a> 2.6 梯度下降三兄弟（BGD，SGD，MBGD）</h2><h3 id="261-批量梯度下降法batch-gradient-descent"><a class="anchor" href="#261-批量梯度下降法batch-gradient-descent">#</a> 2.6.1 批量梯度下降法（Batch Gradient Descent）</h3><p>批量梯度下降法每次都使用训练集中的所有样本更新参数。它得到的是一个全局最优解，但是每迭代一步，都要用到训练集所有的数据，如果 m 很大，那么迭代速度就会变得很慢。<br>优点：可以得出全局最优解。<br>缺点：样本数据集大时，训练速度慢。</p><h3 id="262-随机梯度下降法stochastic-gradient-descent"><a class="anchor" href="#262-随机梯度下降法stochastic-gradient-descent">#</a> 2.6.2 随机梯度下降法（Stochastic Gradient Descent）</h3><p>随机梯度下降法每次更新都从样本随机选择 1 组数据，因此随机梯度下降比批量梯度下降在计算量上会大大减少。SGD 有一个缺点是，其噪音较 BGD 要多，使得 SGD 并不是每次迭代都向着整体最优化方向。而且 SGD 因为每次都是使用一个样本进行迭代，因此最终求得的最优解往往不是全局最优解，而只是局部最优解。但是大的整体的方向是向全局最优解的，最终的结果往往是在全局最优解附近。<br>优点：训练速度较快。<br>缺点：过程杂乱，准确度下降。</p><h3 id="3小批量梯度下降法mini-batch-gradient-descent"><a class="anchor" href="#3小批量梯度下降法mini-batch-gradient-descent">#</a> ③小批量梯度下降法（Mini-batch Gradient Descent）</h3><p>小批量梯度下降法对包含 n 个样本的数据集进行计算。综合了上述两种方法，既保证了训练速度快，又保证了准确度。</p><h1 id="3-波士顿房价预测"><a class="anchor" href="#3-波士顿房价预测">#</a> 3 波士顿房价预测</h1><h2 id="31-代码"><a class="anchor" href="#31-代码">#</a> 3.1 代码</h2><p>房屋价格与面积（数据在下面表格中）：</p><table><thead><tr><th><strong>序号</strong></th><th><strong>面积</strong></th><th><strong>价格</strong></th></tr></thead><tbody><tr><td>1</td><td>150</td><td>6450</td></tr><tr><td>2</td><td>200</td><td>7450</td></tr><tr><td>3</td><td>250</td><td>8450</td></tr><tr><td>4</td><td>300</td><td>9450</td></tr><tr><td>5</td><td>350</td><td>11450</td></tr><tr><td>6</td><td>400</td><td>15450</td></tr><tr><td>7</td><td>600</td><td>18450</td></tr></tbody></table><p>使用梯度下降求解线性回归（求<span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><msub><mi>θ</mi><mn>0</mn></msub></mrow><annotation encoding="application/x-tex">θ_0</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:.84444em;vertical-align:-.15em"></span><span class="mord"><span class="mord mathnormal" style="margin-right:.02778em">θ</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:.30110799999999993em"><span style="top:-2.5500000000000003em;margin-left:-.02778em;margin-right:.05em"><span class="pstrut" style="height:2.7em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">0</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.15em"><span></span></span></span></span></span></span></span></span></span>，<span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><msub><mi>θ</mi><mn>1</mn></msub></mrow><annotation encoding="application/x-tex">θ_1</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:.84444em;vertical-align:-.15em"></span><span class="mord"><span class="mord mathnormal" style="margin-right:.02778em">θ</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:.30110799999999993em"><span style="top:-2.5500000000000003em;margin-left:-.02778em;margin-right:.05em"><span class="pstrut" style="height:2.7em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">1</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.15em"><span></span></span></span></span></span></span></span></span></span>）</p><p><span class="katex-display"><span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML" display="block"><semantics><mrow><msub><mi>h</mi><mi>θ</mi></msub><mo stretchy="false">(</mo><mi>x</mi><mo stretchy="false">)</mo><mo>=</mo><msub><mi>θ</mi><mn>0</mn></msub><mo>+</mo><msub><mi>θ</mi><mn>1</mn></msub><mi>x</mi></mrow><annotation encoding="application/x-tex">h_θ(x)=θ_0+θ_1x</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:1em;vertical-align:-.25em"></span><span class="mord"><span class="mord mathnormal">h</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:.33610799999999996em"><span style="top:-2.5500000000000003em;margin-left:0;margin-right:.05em"><span class="pstrut" style="height:2.7em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight" style="margin-right:.02778em">θ</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.15em"><span></span></span></span></span></span></span><span class="mopen">(</span><span class="mord mathnormal">x</span><span class="mclose">)</span><span class="mspace" style="margin-right:.2777777777777778em"></span><span class="mrel">=</span><span class="mspace" style="margin-right:.2777777777777778em"></span></span><span class="base"><span class="strut" style="height:.84444em;vertical-align:-.15em"></span><span class="mord"><span class="mord mathnormal" style="margin-right:.02778em">θ</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:.30110799999999993em"><span style="top:-2.5500000000000003em;margin-left:-.02778em;margin-right:.05em"><span class="pstrut" style="height:2.7em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">0</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.15em"><span></span></span></span></span></span></span><span class="mspace" style="margin-right:.2222222222222222em"></span><span class="mbin">+</span><span class="mspace" style="margin-right:.2222222222222222em"></span></span><span class="base"><span class="strut" style="height:.84444em;vertical-align:-.15em"></span><span class="mord"><span class="mord mathnormal" style="margin-right:.02778em">θ</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:.30110799999999993em"><span style="top:-2.5500000000000003em;margin-left:-.02778em;margin-right:.05em"><span class="pstrut" style="height:2.7em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">1</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.15em"><span></span></span></span></span></span></span><span class="mord mathnormal">x</span></span></span></span></span></p><figure class="highlight python"><figcaption data-lang="python"></figcaption><table><tr><td data-num="1"></td><td><pre><span class="token comment">#房屋价格与面积</span></pre></td></tr><tr><td data-num="2"></td><td><pre><span class="token comment">#序号：1     2    3    4     5     6     7</span></pre></td></tr><tr><td data-num="3"></td><td><pre><span class="token comment">#面积：150  200  250  300   350   400   600  </span></pre></td></tr><tr><td data-num="4"></td><td><pre><span class="token comment">#价格：6450 7450 8450 9450 11450 15450 18450 </span></pre></td></tr><tr><td data-num="5"></td><td><pre></pre></td></tr><tr><td data-num="6"></td><td><pre><span class="token keyword">import</span> matplotlib<span class="token punctuation">.</span>pyplot <span class="token keyword">as</span> plt</pre></td></tr><tr><td data-num="7"></td><td><pre><span class="token keyword">import</span> matplotlib</pre></td></tr><tr><td data-num="8"></td><td><pre><span class="token keyword">from</span> math <span class="token keyword">import</span> <span class="token builtin">pow</span></pre></td></tr><tr><td data-num="9"></td><td><pre><span class="token keyword">from</span> random <span class="token keyword">import</span> uniform</pre></td></tr><tr><td data-num="10"></td><td><pre><span class="token keyword">import</span> random</pre></td></tr><tr><td data-num="11"></td><td><pre></pre></td></tr><tr><td data-num="12"></td><td><pre>x0 <span class="token operator">=</span> <span class="token punctuation">[</span><span class="token number">150</span><span class="token punctuation">,</span><span class="token number">200</span><span class="token punctuation">,</span><span class="token number">250</span><span class="token punctuation">,</span><span class="token number">300</span><span class="token punctuation">,</span><span class="token number">350</span><span class="token punctuation">,</span><span class="token number">400</span><span class="token punctuation">,</span><span class="token number">600</span><span class="token punctuation">]</span></pre></td></tr><tr><td data-num="13"></td><td><pre>y0 <span class="token operator">=</span> <span class="token punctuation">[</span><span class="token number">6450</span><span class="token punctuation">,</span><span class="token number">7450</span><span class="token punctuation">,</span><span class="token number">8450</span><span class="token punctuation">,</span><span class="token number">9450</span><span class="token punctuation">,</span><span class="token number">11450</span><span class="token punctuation">,</span><span class="token number">15450</span><span class="token punctuation">,</span><span class="token number">18450</span><span class="token punctuation">]</span></pre></td></tr><tr><td data-num="14"></td><td><pre><span class="token comment">#为了方便计算，将所有数据缩小 100 倍</span></pre></td></tr><tr><td data-num="15"></td><td><pre>x <span class="token operator">=</span> <span class="token punctuation">[</span><span class="token number">1.50</span><span class="token punctuation">,</span><span class="token number">2.00</span><span class="token punctuation">,</span><span class="token number">2.50</span><span class="token punctuation">,</span><span class="token number">3.00</span><span class="token punctuation">,</span><span class="token number">3.50</span><span class="token punctuation">,</span><span class="token number">4.00</span><span class="token punctuation">,</span><span class="token number">6.00</span><span class="token punctuation">]</span></pre></td></tr><tr><td data-num="16"></td><td><pre>y <span class="token operator">=</span> <span class="token punctuation">[</span><span class="token number">64.50</span><span class="token punctuation">,</span><span class="token number">74.50</span><span class="token punctuation">,</span><span class="token number">84.50</span><span class="token punctuation">,</span><span class="token number">94.50</span><span class="token punctuation">,</span><span class="token number">114.50</span><span class="token punctuation">,</span><span class="token number">154.50</span><span class="token punctuation">,</span><span class="token number">184.50</span><span class="token punctuation">]</span></pre></td></tr><tr><td data-num="17"></td><td><pre></pre></td></tr><tr><td data-num="18"></td><td><pre></pre></td></tr><tr><td data-num="19"></td><td><pre><span class="token comment">#线性回归函数为 y=theta0+theta1*x</span></pre></td></tr><tr><td data-num="20"></td><td><pre><span class="token comment">#参数定义</span></pre></td></tr><tr><td data-num="21"></td><td><pre>theta0 <span class="token operator">=</span> <span class="token number">0.1</span><span class="token comment">#对 theata0 赋值</span></pre></td></tr><tr><td data-num="22"></td><td><pre>theta1 <span class="token operator">=</span> <span class="token number">0.1</span><span class="token comment">#对 theata1 赋值</span></pre></td></tr><tr><td data-num="23"></td><td><pre>alpha <span class="token operator">=</span> <span class="token number">0.1</span><span class="token comment">#学习率</span></pre></td></tr><tr><td data-num="24"></td><td><pre>m <span class="token operator">=</span> <span class="token builtin">len</span><span class="token punctuation">(</span>x<span class="token punctuation">)</span></pre></td></tr><tr><td data-num="25"></td><td><pre>count0 <span class="token operator">=</span> <span class="token number">0</span></pre></td></tr><tr><td data-num="26"></td><td><pre>theta0_list <span class="token operator">=</span> <span class="token punctuation">[</span><span class="token punctuation">]</span></pre></td></tr><tr><td data-num="27"></td><td><pre>theta1_list <span class="token operator">=</span> <span class="token punctuation">[</span><span class="token punctuation">]</span></pre></td></tr><tr><td data-num="28"></td><td><pre></pre></td></tr><tr><td data-num="29"></td><td><pre><span class="token comment">#使用批量梯度下降法</span></pre></td></tr><tr><td data-num="30"></td><td><pre><span class="token keyword">for</span> num <span class="token keyword">in</span> <span class="token builtin">range</span><span class="token punctuation">(</span><span class="token number">10000</span><span class="token punctuation">)</span><span class="token punctuation">:</span></pre></td></tr><tr><td data-num="31"></td><td><pre>    count0 <span class="token operator">+=</span> <span class="token number">1</span></pre></td></tr><tr><td data-num="32"></td><td><pre>    diss <span class="token operator">=</span> <span class="token number">0</span>   <span class="token comment">#误差</span></pre></td></tr><tr><td data-num="33"></td><td><pre>    deriv0 <span class="token operator">=</span> <span class="token number">0</span> <span class="token comment">#对 theata0 导数</span></pre></td></tr><tr><td data-num="34"></td><td><pre>    deriv1 <span class="token operator">=</span> <span class="token number">0</span> <span class="token comment">#对 theata1 导数</span></pre></td></tr><tr><td data-num="35"></td><td><pre>    <span class="token comment">#求导</span></pre></td></tr><tr><td data-num="36"></td><td><pre>    <span class="token keyword">for</span> i <span class="token keyword">in</span> <span class="token builtin">range</span><span class="token punctuation">(</span>m<span class="token punctuation">)</span><span class="token punctuation">:</span></pre></td></tr><tr><td data-num="37"></td><td><pre>        deriv0 <span class="token operator">+=</span> <span class="token punctuation">(</span>theta0<span class="token operator">+</span>theta1<span class="token operator">*</span>x<span class="token punctuation">[</span>i<span class="token punctuation">]</span><span class="token operator">-</span>y<span class="token punctuation">[</span>i<span class="token punctuation">]</span><span class="token punctuation">)</span><span class="token operator">/</span>m</pre></td></tr><tr><td data-num="38"></td><td><pre>        deriv1 <span class="token operator">+=</span> <span class="token punctuation">(</span><span class="token punctuation">(</span>theta0<span class="token operator">+</span>theta1<span class="token operator">*</span>x<span class="token punctuation">[</span>i<span class="token punctuation">]</span><span class="token operator">-</span>y<span class="token punctuation">[</span>i<span class="token punctuation">]</span><span class="token punctuation">)</span><span class="token operator">/</span>m<span class="token punctuation">)</span><span class="token operator">*</span>x<span class="token punctuation">[</span>i<span class="token punctuation">]</span></pre></td></tr><tr><td data-num="39"></td><td><pre>    <span class="token comment">#更新 theta0 和 theta1</span></pre></td></tr><tr><td data-num="40"></td><td><pre>    <span class="token keyword">for</span> i <span class="token keyword">in</span> <span class="token builtin">range</span><span class="token punctuation">(</span>m<span class="token punctuation">)</span><span class="token punctuation">:</span></pre></td></tr><tr><td data-num="41"></td><td><pre>        theta0 <span class="token operator">=</span> theta0 <span class="token operator">-</span> alpha<span class="token operator">*</span><span class="token punctuation">(</span><span class="token punctuation">(</span>theta0<span class="token operator">+</span>theta1<span class="token operator">*</span>x<span class="token punctuation">[</span>i<span class="token punctuation">]</span><span class="token operator">-</span>y<span class="token punctuation">[</span>i<span class="token punctuation">]</span><span class="token punctuation">)</span><span class="token operator">/</span>m<span class="token punctuation">)</span> </pre></td></tr><tr><td data-num="42"></td><td><pre>        theta1 <span class="token operator">=</span> theta1 <span class="token operator">-</span> alpha<span class="token operator">*</span><span class="token punctuation">(</span><span class="token punctuation">(</span>theta0<span class="token operator">+</span>theta1<span class="token operator">*</span>x<span class="token punctuation">[</span>i<span class="token punctuation">]</span><span class="token operator">-</span>y<span class="token punctuation">[</span>i<span class="token punctuation">]</span><span class="token punctuation">)</span><span class="token operator">/</span>m<span class="token punctuation">)</span><span class="token operator">*</span>x<span class="token punctuation">[</span>i<span class="token punctuation">]</span></pre></td></tr><tr><td data-num="43"></td><td><pre>    <span class="token comment">#求损失函数 J (θ)</span></pre></td></tr><tr><td data-num="44"></td><td><pre>    <span class="token keyword">for</span> i <span class="token keyword">in</span> <span class="token builtin">range</span><span class="token punctuation">(</span>m<span class="token punctuation">)</span><span class="token punctuation">:</span></pre></td></tr><tr><td data-num="45"></td><td><pre>        diss <span class="token operator">=</span> diss <span class="token operator">+</span> <span class="token punctuation">(</span><span class="token number">1</span><span class="token operator">/</span><span class="token punctuation">(</span><span class="token number">2</span><span class="token operator">*</span>m<span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token operator">*</span><span class="token builtin">pow</span><span class="token punctuation">(</span><span class="token punctuation">(</span>theta0<span class="token operator">+</span>theta1<span class="token operator">*</span>x<span class="token punctuation">[</span>i<span class="token punctuation">]</span><span class="token operator">-</span>y<span class="token punctuation">[</span>i<span class="token punctuation">]</span><span class="token punctuation">)</span><span class="token punctuation">,</span><span class="token number">2</span><span class="token punctuation">)</span></pre></td></tr><tr><td data-num="46"></td><td><pre>        </pre></td></tr><tr><td data-num="47"></td><td><pre>    theta0_list<span class="token punctuation">.</span>append<span class="token punctuation">(</span>theta0<span class="token operator">*</span><span class="token number">100</span><span class="token punctuation">)</span></pre></td></tr><tr><td data-num="48"></td><td><pre>    theta1_list<span class="token punctuation">.</span>append<span class="token punctuation">(</span>theta1<span class="token punctuation">)</span></pre></td></tr><tr><td data-num="49"></td><td><pre>    <span class="token comment">#如果误差已经很小，则退出循环</span></pre></td></tr><tr><td data-num="50"></td><td><pre>    <span class="token keyword">if</span> diss <span class="token operator">&lt;=</span> <span class="token number">0.001</span><span class="token punctuation">:</span></pre></td></tr><tr><td data-num="51"></td><td><pre>        <span class="token keyword">break</span></pre></td></tr><tr><td data-num="52"></td><td><pre>    </pre></td></tr><tr><td data-num="53"></td><td><pre>theta0 <span class="token operator">=</span> theta0<span class="token operator">*</span><span class="token number">100</span><span class="token comment">#前面所有数据缩小了 100 倍，所以求出的 theta0 需要放大 100 倍，theta1 不用变</span></pre></td></tr><tr><td data-num="54"></td><td><pre></pre></td></tr><tr><td data-num="55"></td><td><pre><span class="token comment">#使用随机梯度下降法</span></pre></td></tr><tr><td data-num="56"></td><td><pre>theta2 <span class="token operator">=</span> <span class="token number">0.1</span><span class="token comment">#对 theata2 赋值</span></pre></td></tr><tr><td data-num="57"></td><td><pre>theta3 <span class="token operator">=</span> <span class="token number">0.1</span><span class="token comment">#对 theata3 赋值</span></pre></td></tr><tr><td data-num="58"></td><td><pre>count1 <span class="token operator">=</span> <span class="token number">0</span></pre></td></tr><tr><td data-num="59"></td><td><pre>theta2_list <span class="token operator">=</span> <span class="token punctuation">[</span><span class="token punctuation">]</span></pre></td></tr><tr><td data-num="60"></td><td><pre>theta3_list <span class="token operator">=</span> <span class="token punctuation">[</span><span class="token punctuation">]</span></pre></td></tr><tr><td data-num="61"></td><td><pre></pre></td></tr><tr><td data-num="62"></td><td><pre><span class="token keyword">for</span> num <span class="token keyword">in</span> <span class="token builtin">range</span><span class="token punctuation">(</span><span class="token number">10000</span><span class="token punctuation">)</span><span class="token punctuation">:</span></pre></td></tr><tr><td data-num="63"></td><td><pre>    count1 <span class="token operator">+=</span> <span class="token number">1</span></pre></td></tr><tr><td data-num="64"></td><td><pre>    diss <span class="token operator">=</span> <span class="token number">0</span>   <span class="token comment">#误差</span></pre></td></tr><tr><td data-num="65"></td><td><pre>    deriv2 <span class="token operator">=</span> <span class="token number">0</span> <span class="token comment">#对 theata2 导数</span></pre></td></tr><tr><td data-num="66"></td><td><pre>    deriv3 <span class="token operator">=</span> <span class="token number">0</span> <span class="token comment">#对 theata3 导数</span></pre></td></tr><tr><td data-num="67"></td><td><pre>    <span class="token comment">#求导</span></pre></td></tr><tr><td data-num="68"></td><td><pre>    <span class="token keyword">for</span> i <span class="token keyword">in</span> <span class="token builtin">range</span><span class="token punctuation">(</span>m<span class="token punctuation">)</span><span class="token punctuation">:</span></pre></td></tr><tr><td data-num="69"></td><td><pre>        deriv2 <span class="token operator">+=</span> <span class="token punctuation">(</span>theta2<span class="token operator">+</span>theta3<span class="token operator">*</span>x<span class="token punctuation">[</span>i<span class="token punctuation">]</span><span class="token operator">-</span>y<span class="token punctuation">[</span>i<span class="token punctuation">]</span><span class="token punctuation">)</span><span class="token operator">/</span>m</pre></td></tr><tr><td data-num="70"></td><td><pre>        deriv3 <span class="token operator">+=</span> <span class="token punctuation">(</span><span class="token punctuation">(</span>theta2<span class="token operator">+</span>theta3<span class="token operator">*</span>x<span class="token punctuation">[</span>i<span class="token punctuation">]</span><span class="token operator">-</span>y<span class="token punctuation">[</span>i<span class="token punctuation">]</span><span class="token punctuation">)</span><span class="token operator">/</span>m<span class="token punctuation">)</span><span class="token operator">*</span>x<span class="token punctuation">[</span>i<span class="token punctuation">]</span></pre></td></tr><tr><td data-num="71"></td><td><pre>    <span class="token comment">#更新 theta0 和 theta1</span></pre></td></tr><tr><td data-num="72"></td><td><pre>    <span class="token keyword">for</span> i <span class="token keyword">in</span> <span class="token builtin">range</span><span class="token punctuation">(</span>m<span class="token punctuation">)</span><span class="token punctuation">:</span></pre></td></tr><tr><td data-num="73"></td><td><pre>        theta2 <span class="token operator">=</span> theta2 <span class="token operator">-</span> alpha<span class="token operator">*</span><span class="token punctuation">(</span><span class="token punctuation">(</span>theta2<span class="token operator">+</span>theta3<span class="token operator">*</span>x<span class="token punctuation">[</span>i<span class="token punctuation">]</span><span class="token operator">-</span>y<span class="token punctuation">[</span>i<span class="token punctuation">]</span><span class="token punctuation">)</span><span class="token operator">/</span>m<span class="token punctuation">)</span> </pre></td></tr><tr><td data-num="74"></td><td><pre>        theta3 <span class="token operator">=</span> theta3 <span class="token operator">-</span> alpha<span class="token operator">*</span><span class="token punctuation">(</span><span class="token punctuation">(</span>theta2<span class="token operator">+</span>theta3<span class="token operator">*</span>x<span class="token punctuation">[</span>i<span class="token punctuation">]</span><span class="token operator">-</span>y<span class="token punctuation">[</span>i<span class="token punctuation">]</span><span class="token punctuation">)</span><span class="token operator">/</span>m<span class="token punctuation">)</span><span class="token operator">*</span>x<span class="token punctuation">[</span>i<span class="token punctuation">]</span></pre></td></tr><tr><td data-num="75"></td><td><pre>    <span class="token comment">#求损失函数 J (θ)</span></pre></td></tr><tr><td data-num="76"></td><td><pre>    rand_i <span class="token operator">=</span> random<span class="token punctuation">.</span>randrange<span class="token punctuation">(</span><span class="token number">0</span><span class="token punctuation">,</span>m<span class="token punctuation">)</span></pre></td></tr><tr><td data-num="77"></td><td><pre>    diss <span class="token operator">=</span> diss <span class="token operator">+</span> <span class="token punctuation">(</span><span class="token number">1</span><span class="token operator">/</span><span class="token punctuation">(</span><span class="token number">2</span><span class="token operator">*</span>m<span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token operator">*</span><span class="token builtin">pow</span><span class="token punctuation">(</span><span class="token punctuation">(</span>theta2<span class="token operator">+</span>theta3<span class="token operator">*</span>x<span class="token punctuation">[</span>rand_i<span class="token punctuation">]</span><span class="token operator">-</span>y<span class="token punctuation">[</span>rand_i<span class="token punctuation">]</span><span class="token punctuation">)</span><span class="token punctuation">,</span><span class="token number">2</span><span class="token punctuation">)</span></pre></td></tr><tr><td data-num="78"></td><td><pre></pre></td></tr><tr><td data-num="79"></td><td><pre>    theta2_list<span class="token punctuation">.</span>append<span class="token punctuation">(</span>theta2<span class="token operator">*</span><span class="token number">100</span><span class="token punctuation">)</span></pre></td></tr><tr><td data-num="80"></td><td><pre>    theta3_list<span class="token punctuation">.</span>append<span class="token punctuation">(</span>theta3<span class="token punctuation">)</span></pre></td></tr><tr><td data-num="81"></td><td><pre>    <span class="token comment">#如果误差已经很小，则退出循环</span></pre></td></tr><tr><td data-num="82"></td><td><pre>    <span class="token keyword">if</span> diss <span class="token operator">&lt;=</span> <span class="token number">0.001</span><span class="token punctuation">:</span></pre></td></tr><tr><td data-num="83"></td><td><pre>        <span class="token keyword">break</span></pre></td></tr><tr><td data-num="84"></td><td><pre>theta2 <span class="token operator">=</span> theta2<span class="token operator">*</span><span class="token number">100</span></pre></td></tr><tr><td data-num="85"></td><td><pre>    </pre></td></tr><tr><td data-num="86"></td><td><pre><span class="token keyword">print</span><span class="token punctuation">(</span><span class="token string">"批量梯度下降最终得到theta0=&#123;&#125;，theta1=&#123;&#125;"</span><span class="token punctuation">.</span><span class="token builtin">format</span><span class="token punctuation">(</span>theta0<span class="token punctuation">,</span>theta1<span class="token punctuation">)</span><span class="token punctuation">)</span></pre></td></tr><tr><td data-num="87"></td><td><pre><span class="token keyword">print</span><span class="token punctuation">(</span><span class="token string">"           得到的回归函数是：y=&#123;&#125;+&#123;&#125;*x"</span><span class="token punctuation">.</span><span class="token builtin">format</span><span class="token punctuation">(</span>theta0<span class="token punctuation">,</span>theta1<span class="token punctuation">)</span><span class="token punctuation">)</span></pre></td></tr><tr><td data-num="88"></td><td><pre><span class="token keyword">print</span><span class="token punctuation">(</span><span class="token string">"随机梯度下降最终得到theta0=&#123;&#125;，theta1=&#123;&#125;"</span><span class="token punctuation">.</span><span class="token builtin">format</span><span class="token punctuation">(</span>theta2<span class="token punctuation">,</span>theta3<span class="token punctuation">)</span><span class="token punctuation">)</span></pre></td></tr><tr><td data-num="89"></td><td><pre><span class="token keyword">print</span><span class="token punctuation">(</span><span class="token string">"           得到的回归函数是：y=&#123;&#125;+&#123;&#125;*x"</span><span class="token punctuation">.</span><span class="token builtin">format</span><span class="token punctuation">(</span>theta2<span class="token punctuation">,</span>theta3<span class="token punctuation">)</span><span class="token punctuation">)</span></pre></td></tr><tr><td data-num="90"></td><td><pre><span class="token comment">#画原始数据图和函数图</span></pre></td></tr><tr><td data-num="91"></td><td><pre>matplotlib<span class="token punctuation">.</span>rcParams<span class="token punctuation">[</span><span class="token string">'font.sans-serif'</span><span class="token punctuation">]</span> <span class="token operator">=</span> <span class="token punctuation">[</span><span class="token string">'SimHei'</span><span class="token punctuation">]</span></pre></td></tr><tr><td data-num="92"></td><td><pre>plt<span class="token punctuation">.</span>plot<span class="token punctuation">(</span>x0<span class="token punctuation">,</span>y0<span class="token punctuation">,</span><span class="token string">'bo'</span><span class="token punctuation">,</span>label<span class="token operator">=</span><span class="token string">'数据'</span><span class="token punctuation">,</span>color<span class="token operator">=</span><span class="token string">'black'</span><span class="token punctuation">)</span></pre></td></tr><tr><td data-num="93"></td><td><pre>plt<span class="token punctuation">.</span>plot<span class="token punctuation">(</span>x0<span class="token punctuation">,</span><span class="token punctuation">[</span>theta0<span class="token operator">+</span>theta1<span class="token operator">*</span>x <span class="token keyword">for</span> x <span class="token keyword">in</span> x0<span class="token punctuation">]</span><span class="token punctuation">,</span>label<span class="token operator">=</span><span class="token string">'批量梯度下降'</span><span class="token punctuation">,</span>color<span class="token operator">=</span><span class="token string">'red'</span><span class="token punctuation">)</span></pre></td></tr><tr><td data-num="94"></td><td><pre>plt<span class="token punctuation">.</span>plot<span class="token punctuation">(</span>x0<span class="token punctuation">,</span><span class="token punctuation">[</span>theta2<span class="token operator">+</span>theta3<span class="token operator">*</span>x <span class="token keyword">for</span> x <span class="token keyword">in</span> x0<span class="token punctuation">]</span><span class="token punctuation">,</span>label<span class="token operator">=</span><span class="token string">'随机梯度下降'</span><span class="token punctuation">,</span>color<span class="token operator">=</span><span class="token string">'blue'</span><span class="token punctuation">)</span></pre></td></tr><tr><td data-num="95"></td><td><pre>plt<span class="token punctuation">.</span>xlabel<span class="token punctuation">(</span><span class="token string">'x（面积）'</span><span class="token punctuation">)</span></pre></td></tr><tr><td data-num="96"></td><td><pre>plt<span class="token punctuation">.</span>ylabel<span class="token punctuation">(</span><span class="token string">'y（价格）'</span><span class="token punctuation">)</span></pre></td></tr><tr><td data-num="97"></td><td><pre>plt<span class="token punctuation">.</span>legend<span class="token punctuation">(</span><span class="token punctuation">)</span></pre></td></tr><tr><td data-num="98"></td><td><pre>plt<span class="token punctuation">.</span>show<span class="token punctuation">(</span><span class="token punctuation">)</span></pre></td></tr><tr><td data-num="99"></td><td><pre>plt<span class="token punctuation">.</span>scatter<span class="token punctuation">(</span><span class="token builtin">range</span><span class="token punctuation">(</span>count0<span class="token punctuation">)</span><span class="token punctuation">,</span>theta0_list<span class="token punctuation">,</span>s<span class="token operator">=</span><span class="token number">1</span><span class="token punctuation">)</span></pre></td></tr><tr><td data-num="100"></td><td><pre>plt<span class="token punctuation">.</span>scatter<span class="token punctuation">(</span><span class="token builtin">range</span><span class="token punctuation">(</span>count0<span class="token punctuation">)</span><span class="token punctuation">,</span>theta1_list<span class="token punctuation">,</span>s<span class="token operator">=</span><span class="token number">1</span><span class="token punctuation">)</span></pre></td></tr><tr><td data-num="101"></td><td><pre>plt<span class="token punctuation">.</span>xlabel<span class="token punctuation">(</span><span class="token string">'上方为theta0，下方为theta1'</span><span class="token punctuation">)</span></pre></td></tr><tr><td data-num="102"></td><td><pre>plt<span class="token punctuation">.</span>show<span class="token punctuation">(</span><span class="token punctuation">)</span></pre></td></tr><tr><td data-num="103"></td><td><pre>plt<span class="token punctuation">.</span>scatter<span class="token punctuation">(</span><span class="token builtin">range</span><span class="token punctuation">(</span>count1<span class="token punctuation">)</span><span class="token punctuation">,</span>theta2_list<span class="token punctuation">,</span>s<span class="token operator">=</span><span class="token number">3</span><span class="token punctuation">)</span></pre></td></tr><tr><td data-num="104"></td><td><pre>plt<span class="token punctuation">.</span>scatter<span class="token punctuation">(</span><span class="token builtin">range</span><span class="token punctuation">(</span>count1<span class="token punctuation">)</span><span class="token punctuation">,</span>theta3_list<span class="token punctuation">,</span>s<span class="token operator">=</span><span class="token number">3</span><span class="token punctuation">)</span></pre></td></tr><tr><td data-num="105"></td><td><pre>plt<span class="token punctuation">.</span>xlabel<span class="token punctuation">(</span><span class="token string">'上方为theta0，下方为theta1'</span><span class="token punctuation">)</span></pre></td></tr><tr><td data-num="106"></td><td><pre>plt<span class="token punctuation">.</span>show<span class="token punctuation">(</span><span class="token punctuation">)</span></pre></td></tr></table></figure><h2 id="32-输出结果"><a class="anchor" href="#32-输出结果">#</a> 3.2 输出结果：</h2><pre><code>批量梯度下降最终得到theta0=2019.4181661097477，theta1=28.06952868382116
           得到的回归函数是：y=2019.4181661097477+28.06952868382116*x  
随机梯度下降最终得到theta0=1685.432918583438，theta1=28.789137185954317
           得到的回归函数是：y=1685.432918583438+28.789137185954317*x  
</code></pre><p>线性回归函数图像：</p><p><img data-src="https://shun309.oss-cn-hangzhou.aliyuncs.com/photos/20210415223448.png" alt=""></p><p>批量梯度下降时的 theta0 和 theta1 的变化：</p><p><img data-src="https://shun309.oss-cn-hangzhou.aliyuncs.com/photos/20210415223533.png" alt=""></p><p>随机梯度下降时的 theta0 和 theta1 的变化：</p><p><img data-src="https://shun309.oss-cn-hangzhou.aliyuncs.com/photos/20210415223635.png" alt=""></p><h1 id="4-多元线性回归及梯度下降"><a class="anchor" href="#4-多元线性回归及梯度下降">#</a> 4 多元线性回归及梯度下降</h1><h2 id="41-定义数据"><a class="anchor" href="#41-定义数据">#</a> 4.1 定义数据</h2><p>以下面一组数据作为例子：</p><p><img data-src="https://shun309.oss-cn-hangzhou.aliyuncs.com/photos/20210415230137.png" alt=""></p><p>以<strong>上</strong>角标作为<strong>行</strong>索引，以<strong>下</strong>角标作为<strong>列</strong>索引</p><p>第二行可以写成：</p><p><span class="katex-display"><span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML" display="block"><semantics><mrow><msup><mi>x</mi><mrow><mo stretchy="false">(</mo><mn>2</mn><mo stretchy="false">)</mo></mrow></msup><mo>=</mo><mrow><mo fence="true">[</mo><mtable rowspacing="0.15999999999999992em" columnspacing="1em"><mtr><mtd><mstyle scriptlevel="0" displaystyle="false"><mn>916</mn></mstyle></mtd></mtr><mtr><mtd><mstyle scriptlevel="0" displaystyle="false"><mn>1201</mn></mstyle></mtd></mtr><mtr><mtd><mstyle scriptlevel="0" displaystyle="false"><mn>5</mn></mstyle></mtd></mtr><mtr><mtd><mstyle scriptlevel="0" displaystyle="false"><mo lspace="0em" rspace="0em">…</mo></mstyle></mtd></mtr><mtr><mtd><mstyle scriptlevel="0" displaystyle="false"><mn>33</mn></mstyle></mtd></mtr></mtable><mo fence="true">]</mo></mrow></mrow><annotation encoding="application/x-tex">x^{(2)}=\begin{bmatrix} 916 \\ 1201 \\ 5 \\ … \\ 33 \end{bmatrix}</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:.938em;vertical-align:0"></span><span class="mord"><span class="mord mathnormal">x</span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height:.938em"><span style="top:-3.113em;margin-right:.05em"><span class="pstrut" style="height:2.7em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mopen mtight">(</span><span class="mord mtight">2</span><span class="mclose mtight">)</span></span></span></span></span></span></span></span></span><span class="mspace" style="margin-right:.2777777777777778em"></span><span class="mrel">=</span><span class="mspace" style="margin-right:.2777777777777778em"></span></span><span class="base"><span class="strut" style="height:6.00503em;vertical-align:-2.75004em"></span><span class="minner"><span class="mopen"><span class="delimsizing mult"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:3.2549900000000003em"><span style="top:-1.0499800000000006em"><span class="pstrut" style="height:3.1550000000000002em"></span><span class="delimsizinginner delim-size4"><span>⎣</span></span></span><span style="top:-2.1999800000000005em"><span class="pstrut" style="height:3.1550000000000002em"></span><span class="delimsizinginner delim-size4"><span>⎢</span></span></span><span style="top:-2.79598em"><span class="pstrut" style="height:3.1550000000000002em"></span><span class="delimsizinginner delim-size4"><span>⎢</span></span></span><span style="top:-3.39198em"><span class="pstrut" style="height:3.1550000000000002em"></span><span class="delimsizinginner delim-size4"><span>⎢</span></span></span><span style="top:-3.9879800000000003em"><span class="pstrut" style="height:3.1550000000000002em"></span><span class="delimsizinginner delim-size4"><span>⎢</span></span></span><span style="top:-4.0139700000000005em"><span class="pstrut" style="height:3.1550000000000002em"></span><span class="delimsizinginner delim-size4"><span>⎢</span></span></span><span style="top:-5.25499em"><span class="pstrut" style="height:3.1550000000000002em"></span><span class="delimsizinginner delim-size4"><span>⎡</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:2.75004em"><span></span></span></span></span></span></span><span class="mord"><span class="mtable"><span class="col-align-c"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:3.2500000000000004em"><span style="top:-5.410000000000001em"><span class="pstrut" style="height:3em"></span><span class="mord"><span class="mord">9</span><span class="mord">1</span><span class="mord">6</span></span></span><span style="top:-4.21em"><span class="pstrut" style="height:3em"></span><span class="mord"><span class="mord">1</span><span class="mord">2</span><span class="mord">0</span><span class="mord">1</span></span></span><span style="top:-3.01em"><span class="pstrut" style="height:3em"></span><span class="mord"><span class="mord">5</span></span></span><span style="top:-1.8099999999999998em"><span class="pstrut" style="height:3em"></span><span class="mord"><span class="minner">…</span></span></span><span style="top:-.6099999999999997em"><span class="pstrut" style="height:3em"></span><span class="mord"><span class="mord">3</span><span class="mord">3</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:2.7500000000000004em"><span></span></span></span></span></span></span></span><span class="mclose"><span class="delimsizing mult"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:3.2549900000000003em"><span style="top:-1.0499800000000006em"><span class="pstrut" style="height:3.1550000000000002em"></span><span class="delimsizinginner delim-size4"><span>⎦</span></span></span><span style="top:-2.1999800000000005em"><span class="pstrut" style="height:3.1550000000000002em"></span><span class="delimsizinginner delim-size4"><span>⎥</span></span></span><span style="top:-2.79598em"><span class="pstrut" style="height:3.1550000000000002em"></span><span class="delimsizinginner delim-size4"><span>⎥</span></span></span><span style="top:-3.39198em"><span class="pstrut" style="height:3.1550000000000002em"></span><span class="delimsizinginner delim-size4"><span>⎥</span></span></span><span style="top:-3.9879800000000003em"><span class="pstrut" style="height:3.1550000000000002em"></span><span class="delimsizinginner delim-size4"><span>⎥</span></span></span><span style="top:-4.0139700000000005em"><span class="pstrut" style="height:3.1550000000000002em"></span><span class="delimsizinginner delim-size4"><span>⎥</span></span></span><span style="top:-5.25499em"><span class="pstrut" style="height:3.1550000000000002em"></span><span class="delimsizinginner delim-size4"><span>⎤</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:2.75004em"><span></span></span></span></span></span></span></span></span></span></span></span></p><p>位于第二行第一列位置的数写成：</p><p><span class="katex-display"><span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML" display="block"><semantics><mrow><msubsup><mi>x</mi><mn>1</mn><mrow><mo stretchy="false">(</mo><mn>2</mn><mo stretchy="false">)</mo></mrow></msubsup><mo>=</mo><mn>916</mn></mrow><annotation encoding="application/x-tex">x^{(2)}_1=916</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:1.311108em;vertical-align:-.266308em"></span><span class="mord"><span class="mord mathnormal">x</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:1.0448em"><span style="top:-2.433692em;margin-left:0;margin-right:.05em"><span class="pstrut" style="height:2.7em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">1</span></span></span><span style="top:-3.2198em;margin-right:.05em"><span class="pstrut" style="height:2.7em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mopen mtight">(</span><span class="mord mtight">2</span><span class="mclose mtight">)</span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.266308em"><span></span></span></span></span></span></span><span class="mspace" style="margin-right:.2777777777777778em"></span><span class="mrel">=</span><span class="mspace" style="margin-right:.2777777777777778em"></span></span><span class="base"><span class="strut" style="height:.64444em;vertical-align:0"></span><span class="mord">9</span><span class="mord">1</span><span class="mord">6</span></span></span></span></span></p><p>以上下角标来区分位置，以便于后期运算。</p><h2 id="42-定义函数"><a class="anchor" href="#42-定义函数">#</a> 4.2 定义函数</h2><p>设置一个回归方程：</p><p><span class="katex-display"><span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML" display="block"><semantics><mrow><msub><mi>h</mi><mi>θ</mi></msub><mo stretchy="false">(</mo><mi>x</mi><mo stretchy="false">)</mo><mo>=</mo><msub><mi>θ</mi><mn>0</mn></msub><mo>+</mo><msub><mi>θ</mi><mn>1</mn></msub><msub><mi>x</mi><mn>1</mn></msub><mo>+</mo><msub><mi>θ</mi><mn>2</mn></msub><msub><mi>x</mi><mn>2</mn></msub><mo>+</mo><msub><mi>θ</mi><mn>3</mn></msub><msub><mi>x</mi><mn>3</mn></msub><mo>+</mo><mo>⋯</mo><mo>+</mo><msub><mi>θ</mi><mi>n</mi></msub><msub><mi>x</mi><mi>n</mi></msub></mrow><annotation encoding="application/x-tex">h _\theta(x)=\theta_0+\theta_1x_1+\theta_2x_2+\theta_3x_3+\cdots+\theta_nx_n</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:1em;vertical-align:-.25em"></span><span class="mord"><span class="mord mathnormal">h</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:.33610799999999996em"><span style="top:-2.5500000000000003em;margin-left:0;margin-right:.05em"><span class="pstrut" style="height:2.7em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight" style="margin-right:.02778em">θ</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.15em"><span></span></span></span></span></span></span><span class="mopen">(</span><span class="mord mathnormal">x</span><span class="mclose">)</span><span class="mspace" style="margin-right:.2777777777777778em"></span><span class="mrel">=</span><span class="mspace" style="margin-right:.2777777777777778em"></span></span><span class="base"><span class="strut" style="height:.84444em;vertical-align:-.15em"></span><span class="mord"><span class="mord mathnormal" style="margin-right:.02778em">θ</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:.30110799999999993em"><span style="top:-2.5500000000000003em;margin-left:-.02778em;margin-right:.05em"><span class="pstrut" style="height:2.7em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">0</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.15em"><span></span></span></span></span></span></span><span class="mspace" style="margin-right:.2222222222222222em"></span><span class="mbin">+</span><span class="mspace" style="margin-right:.2222222222222222em"></span></span><span class="base"><span class="strut" style="height:.84444em;vertical-align:-.15em"></span><span class="mord"><span class="mord mathnormal" style="margin-right:.02778em">θ</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:.30110799999999993em"><span style="top:-2.5500000000000003em;margin-left:-.02778em;margin-right:.05em"><span class="pstrut" style="height:2.7em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">1</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.15em"><span></span></span></span></span></span></span><span class="mord"><span class="mord mathnormal">x</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:.30110799999999993em"><span style="top:-2.5500000000000003em;margin-left:0;margin-right:.05em"><span class="pstrut" style="height:2.7em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">1</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.15em"><span></span></span></span></span></span></span><span class="mspace" style="margin-right:.2222222222222222em"></span><span class="mbin">+</span><span class="mspace" style="margin-right:.2222222222222222em"></span></span><span class="base"><span class="strut" style="height:.84444em;vertical-align:-.15em"></span><span class="mord"><span class="mord mathnormal" style="margin-right:.02778em">θ</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:.30110799999999993em"><span style="top:-2.5500000000000003em;margin-left:-.02778em;margin-right:.05em"><span class="pstrut" style="height:2.7em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">2</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.15em"><span></span></span></span></span></span></span><span class="mord"><span class="mord mathnormal">x</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:.30110799999999993em"><span style="top:-2.5500000000000003em;margin-left:0;margin-right:.05em"><span class="pstrut" style="height:2.7em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">2</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.15em"><span></span></span></span></span></span></span><span class="mspace" style="margin-right:.2222222222222222em"></span><span class="mbin">+</span><span class="mspace" style="margin-right:.2222222222222222em"></span></span><span class="base"><span class="strut" style="height:.84444em;vertical-align:-.15em"></span><span class="mord"><span class="mord mathnormal" style="margin-right:.02778em">θ</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:.30110799999999993em"><span style="top:-2.5500000000000003em;margin-left:-.02778em;margin-right:.05em"><span class="pstrut" style="height:2.7em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">3</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.15em"><span></span></span></span></span></span></span><span class="mord"><span class="mord mathnormal">x</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:.30110799999999993em"><span style="top:-2.5500000000000003em;margin-left:0;margin-right:.05em"><span class="pstrut" style="height:2.7em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">3</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.15em"><span></span></span></span></span></span></span><span class="mspace" style="margin-right:.2222222222222222em"></span><span class="mbin">+</span><span class="mspace" style="margin-right:.2222222222222222em"></span></span><span class="base"><span class="strut" style="height:.66666em;vertical-align:-.08333em"></span><span class="minner">⋯</span><span class="mspace" style="margin-right:.2222222222222222em"></span><span class="mbin">+</span><span class="mspace" style="margin-right:.2222222222222222em"></span></span><span class="base"><span class="strut" style="height:.84444em;vertical-align:-.15em"></span><span class="mord"><span class="mord mathnormal" style="margin-right:.02778em">θ</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:.151392em"><span style="top:-2.5500000000000003em;margin-left:-.02778em;margin-right:.05em"><span class="pstrut" style="height:2.7em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight">n</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.15em"><span></span></span></span></span></span></span><span class="mord"><span class="mord mathnormal">x</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:.151392em"><span style="top:-2.5500000000000003em;margin-left:0;margin-right:.05em"><span class="pstrut" style="height:2.7em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight">n</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.15em"><span></span></span></span></span></span></span></span></span></span></span></p><p>添加一个列向量</p><p><span class="katex-display"><span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML" display="block"><semantics><mrow><msub><mi>x</mi><mn>0</mn></msub><mo>=</mo><msup><mrow><mo fence="true">[</mo><mtable rowspacing="0.15999999999999992em" columnspacing="1em"><mtr><mtd><mstyle scriptlevel="0" displaystyle="false"><mn>1</mn></mstyle></mtd><mtd><mstyle scriptlevel="0" displaystyle="false"><mn>1</mn></mstyle></mtd><mtd><mstyle scriptlevel="0" displaystyle="false"><mn>1</mn></mstyle></mtd><mtd><mstyle scriptlevel="0" displaystyle="false"><mn>1</mn></mstyle></mtd><mtd><mstyle scriptlevel="0" displaystyle="false"><mo lspace="0em" rspace="0em">…</mo></mstyle></mtd><mtd><mstyle scriptlevel="0" displaystyle="false"><mn>1</mn></mstyle></mtd></mtr></mtable><mo fence="true">]</mo></mrow><mi>T</mi></msup></mrow><annotation encoding="application/x-tex">x_0=\begin{bmatrix} 1 &amp; 1 &amp; 1 &amp; 1 &amp; … &amp; 1\end{bmatrix}^T</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:.58056em;vertical-align:-.15em"></span><span class="mord"><span class="mord mathnormal">x</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:.30110799999999993em"><span style="top:-2.5500000000000003em;margin-left:0;margin-right:.05em"><span class="pstrut" style="height:2.7em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">0</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.15em"><span></span></span></span></span></span></span><span class="mspace" style="margin-right:.2777777777777778em"></span><span class="mrel">=</span><span class="mspace" style="margin-right:.2777777777777778em"></span></span><span class="base"><span class="strut" style="height:1.431241em;vertical-align:-.35001em"></span><span class="minner"><span class="minner"><span class="mopen delimcenter" style="top:0"><span class="delimsizing size1">[</span></span><span class="mord"><span class="mtable"><span class="col-align-c"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:.8500000000000001em"><span style="top:-3.01em"><span class="pstrut" style="height:3em"></span><span class="mord"><span class="mord">1</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.35000000000000003em"><span></span></span></span></span></span><span class="arraycolsep" style="width:.5em"></span><span class="arraycolsep" style="width:.5em"></span><span class="col-align-c"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:.8500000000000001em"><span style="top:-3.01em"><span class="pstrut" style="height:3em"></span><span class="mord"><span class="mord">1</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.35000000000000003em"><span></span></span></span></span></span><span class="arraycolsep" style="width:.5em"></span><span class="arraycolsep" style="width:.5em"></span><span class="col-align-c"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:.8500000000000001em"><span style="top:-3.01em"><span class="pstrut" style="height:3em"></span><span class="mord"><span class="mord">1</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.35000000000000003em"><span></span></span></span></span></span><span class="arraycolsep" style="width:.5em"></span><span class="arraycolsep" style="width:.5em"></span><span class="col-align-c"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:.8500000000000001em"><span style="top:-3.01em"><span class="pstrut" style="height:3em"></span><span class="mord"><span class="mord">1</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.35000000000000003em"><span></span></span></span></span></span><span class="arraycolsep" style="width:.5em"></span><span class="arraycolsep" style="width:.5em"></span><span class="col-align-c"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:.8500000000000001em"><span style="top:-3.01em"><span class="pstrut" style="height:3em"></span><span class="mord"><span class="minner">…</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.35000000000000003em"><span></span></span></span></span></span><span class="arraycolsep" style="width:.5em"></span><span class="arraycolsep" style="width:.5em"></span><span class="col-align-c"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:.8500000000000001em"><span style="top:-3.01em"><span class="pstrut" style="height:3em"></span><span class="mord"><span class="mord">1</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.35000000000000003em"><span></span></span></span></span></span></span></span><span class="mclose delimcenter" style="top:0"><span class="delimsizing size1">]</span></span></span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height:1.081231em"><span style="top:-3.3029em;margin-right:.05em"><span class="pstrut" style="height:2.7em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight" style="margin-right:.13889em">T</span></span></span></span></span></span></span></span></span></span></span></span></p><p>这样方程可以写为：</p><p><span class="katex-display"><span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML" display="block"><semantics><mrow><msub><mi>h</mi><mi>θ</mi></msub><mo stretchy="false">(</mo><mi>x</mi><mo stretchy="false">)</mo><mo>=</mo><msub><mi>θ</mi><mn>0</mn></msub><msub><mi>x</mi><mn>0</mn></msub><mo>+</mo><msub><mi>θ</mi><mn>1</mn></msub><msub><mi>x</mi><mn>1</mn></msub><mo>+</mo><msub><mi>θ</mi><mn>2</mn></msub><msub><mi>x</mi><mn>2</mn></msub><mo>+</mo><msub><mi>θ</mi><mn>3</mn></msub><msub><mi>x</mi><mn>3</mn></msub><mo>+</mo><mo>⋯</mo><mo>+</mo><msub><mi>θ</mi><mi>n</mi></msub><msub><mi>x</mi><mi>n</mi></msub></mrow><annotation encoding="application/x-tex">h_\theta(x)=\theta_0x_0+\theta_1x_1+\theta_2x_2+\theta_3x_3+\cdots+\theta_nx_n</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:1em;vertical-align:-.25em"></span><span class="mord"><span class="mord mathnormal">h</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:.33610799999999996em"><span style="top:-2.5500000000000003em;margin-left:0;margin-right:.05em"><span class="pstrut" style="height:2.7em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight" style="margin-right:.02778em">θ</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.15em"><span></span></span></span></span></span></span><span class="mopen">(</span><span class="mord mathnormal">x</span><span class="mclose">)</span><span class="mspace" style="margin-right:.2777777777777778em"></span><span class="mrel">=</span><span class="mspace" style="margin-right:.2777777777777778em"></span></span><span class="base"><span class="strut" style="height:.84444em;vertical-align:-.15em"></span><span class="mord"><span class="mord mathnormal" style="margin-right:.02778em">θ</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:.30110799999999993em"><span style="top:-2.5500000000000003em;margin-left:-.02778em;margin-right:.05em"><span class="pstrut" style="height:2.7em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">0</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.15em"><span></span></span></span></span></span></span><span class="mord"><span class="mord mathnormal">x</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:.30110799999999993em"><span style="top:-2.5500000000000003em;margin-left:0;margin-right:.05em"><span class="pstrut" style="height:2.7em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">0</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.15em"><span></span></span></span></span></span></span><span class="mspace" style="margin-right:.2222222222222222em"></span><span class="mbin">+</span><span class="mspace" style="margin-right:.2222222222222222em"></span></span><span class="base"><span class="strut" style="height:.84444em;vertical-align:-.15em"></span><span class="mord"><span class="mord mathnormal" style="margin-right:.02778em">θ</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:.30110799999999993em"><span style="top:-2.5500000000000003em;margin-left:-.02778em;margin-right:.05em"><span class="pstrut" style="height:2.7em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">1</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.15em"><span></span></span></span></span></span></span><span class="mord"><span class="mord mathnormal">x</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:.30110799999999993em"><span style="top:-2.5500000000000003em;margin-left:0;margin-right:.05em"><span class="pstrut" style="height:2.7em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">1</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.15em"><span></span></span></span></span></span></span><span class="mspace" style="margin-right:.2222222222222222em"></span><span class="mbin">+</span><span class="mspace" style="margin-right:.2222222222222222em"></span></span><span class="base"><span class="strut" style="height:.84444em;vertical-align:-.15em"></span><span class="mord"><span class="mord mathnormal" style="margin-right:.02778em">θ</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:.30110799999999993em"><span style="top:-2.5500000000000003em;margin-left:-.02778em;margin-right:.05em"><span class="pstrut" style="height:2.7em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">2</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.15em"><span></span></span></span></span></span></span><span class="mord"><span class="mord mathnormal">x</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:.30110799999999993em"><span style="top:-2.5500000000000003em;margin-left:0;margin-right:.05em"><span class="pstrut" style="height:2.7em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">2</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.15em"><span></span></span></span></span></span></span><span class="mspace" style="margin-right:.2222222222222222em"></span><span class="mbin">+</span><span class="mspace" style="margin-right:.2222222222222222em"></span></span><span class="base"><span class="strut" style="height:.84444em;vertical-align:-.15em"></span><span class="mord"><span class="mord mathnormal" style="margin-right:.02778em">θ</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:.30110799999999993em"><span style="top:-2.5500000000000003em;margin-left:-.02778em;margin-right:.05em"><span class="pstrut" style="height:2.7em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">3</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.15em"><span></span></span></span></span></span></span><span class="mord"><span class="mord mathnormal">x</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:.30110799999999993em"><span style="top:-2.5500000000000003em;margin-left:0;margin-right:.05em"><span class="pstrut" style="height:2.7em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">3</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.15em"><span></span></span></span></span></span></span><span class="mspace" style="margin-right:.2222222222222222em"></span><span class="mbin">+</span><span class="mspace" style="margin-right:.2222222222222222em"></span></span><span class="base"><span class="strut" style="height:.66666em;vertical-align:-.08333em"></span><span class="minner">⋯</span><span class="mspace" style="margin-right:.2222222222222222em"></span><span class="mbin">+</span><span class="mspace" style="margin-right:.2222222222222222em"></span></span><span class="base"><span class="strut" style="height:.84444em;vertical-align:-.15em"></span><span class="mord"><span class="mord mathnormal" style="margin-right:.02778em">θ</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:.151392em"><span style="top:-2.5500000000000003em;margin-left:-.02778em;margin-right:.05em"><span class="pstrut" style="height:2.7em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight">n</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.15em"><span></span></span></span></span></span></span><span class="mord"><span class="mord mathnormal">x</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:.151392em"><span style="top:-2.5500000000000003em;margin-left:0;margin-right:.05em"><span class="pstrut" style="height:2.7em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight">n</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.15em"><span></span></span></span></span></span></span></span></span></span></span></p><p>既不会影响到方程的结果，而且使 <span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>x</mi></mrow><annotation encoding="application/x-tex">x</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:.43056em;vertical-align:0"></span><span class="mord mathnormal">x</span></span></span></span> 与 <span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>θ</mi></mrow><annotation encoding="application/x-tex">\theta</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:.69444em;vertical-align:0"></span><span class="mord mathnormal" style="margin-right:.02778em">θ</span></span></span></span> 的数量一致以便于矩阵计算。</p><p>为了更方便表达，分别记为：</p><p><span class="katex-display"><span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML" display="block"><semantics><mrow><mi>X</mi><mo>=</mo><mrow><mo fence="true">[</mo><mtable rowspacing="0.15999999999999992em" columnspacing="1em"><mtr><mtd><mstyle scriptlevel="0" displaystyle="false"><msub><mi>x</mi><mn>0</mn></msub></mstyle></mtd></mtr><mtr><mtd><mstyle scriptlevel="0" displaystyle="false"><msub><mi>x</mi><mn>1</mn></msub></mstyle></mtd></mtr><mtr><mtd><mstyle scriptlevel="0" displaystyle="false"><msub><mi>x</mi><mn>2</mn></msub></mstyle></mtd></mtr><mtr><mtd><mstyle scriptlevel="0" displaystyle="false"><msub><mi>x</mi><mn>3</mn></msub></mstyle></mtd></mtr><mtr><mtd><mstyle scriptlevel="0" displaystyle="false"><mo lspace="0em" rspace="0em">…</mo></mstyle></mtd></mtr><mtr><mtd><mstyle scriptlevel="0" displaystyle="false"><msub><mi>x</mi><mi>n</mi></msub></mstyle></mtd></mtr></mtable><mo fence="true">]</mo></mrow><mo separator="true">,</mo><mi>θ</mi><mo>=</mo><mrow><mo fence="true">[</mo><mtable rowspacing="0.15999999999999992em" columnspacing="1em"><mtr><mtd><mstyle scriptlevel="0" displaystyle="false"><msub><mi>θ</mi><mn>0</mn></msub></mstyle></mtd></mtr><mtr><mtd><mstyle scriptlevel="0" displaystyle="false"><msub><mi>θ</mi><mn>1</mn></msub></mstyle></mtd></mtr><mtr><mtd><mstyle scriptlevel="0" displaystyle="false"><msub><mi>θ</mi><mn>2</mn></msub></mstyle></mtd></mtr><mtr><mtd><mstyle scriptlevel="0" displaystyle="false"><msub><mi>θ</mi><mn>3</mn></msub></mstyle></mtd></mtr><mtr><mtd><mstyle scriptlevel="0" displaystyle="false"><mo lspace="0em" rspace="0em">…</mo></mstyle></mtd></mtr><mtr><mtd><mstyle scriptlevel="0" displaystyle="false"><msub><mi>θ</mi><mi>n</mi></msub></mstyle></mtd></mtr></mtable><mo fence="true">]</mo></mrow></mrow><annotation encoding="application/x-tex">X=\begin{bmatrix} x_0 \\ x_1 \\ x_2 \\ x_3 \\ … \\ x_n \end{bmatrix},\theta=\begin{bmatrix} \theta_0 \\ \theta_1 \\ \theta_2 \\ \theta_3 \\ … \\ \theta_n \end{bmatrix}</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:.68333em;vertical-align:0"></span><span class="mord mathnormal" style="margin-right:.07847em">X</span><span class="mspace" style="margin-right:.2777777777777778em"></span><span class="mrel">=</span><span class="mspace" style="margin-right:.2777777777777778em"></span></span><span class="base"><span class="strut" style="height:7.20703em;vertical-align:-3.3500499999999995em"></span><span class="minner"><span class="mopen"><span class="delimsizing mult"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:3.8569800000000005em"><span style="top:-.44997000000000076em"><span class="pstrut" style="height:3.1550000000000002em"></span><span class="delimsizinginner delim-size4"><span>⎣</span></span></span><span style="top:-1.5999700000000008em"><span class="pstrut" style="height:3.1550000000000002em"></span><span class="delimsizinginner delim-size4"><span>⎢</span></span></span><span style="top:-2.195970000000001em"><span class="pstrut" style="height:3.1550000000000002em"></span><span class="delimsizinginner delim-size4"><span>⎢</span></span></span><span style="top:-2.791970000000001em"><span class="pstrut" style="height:3.1550000000000002em"></span><span class="delimsizinginner delim-size4"><span>⎢</span></span></span><span style="top:-3.3879700000000006em"><span class="pstrut" style="height:3.1550000000000002em"></span><span class="delimsizinginner delim-size4"><span>⎢</span></span></span><span style="top:-3.9839700000000007em"><span class="pstrut" style="height:3.1550000000000002em"></span><span class="delimsizinginner delim-size4"><span>⎢</span></span></span><span style="top:-4.579970000000001em"><span class="pstrut" style="height:3.1550000000000002em"></span><span class="delimsizinginner delim-size4"><span>⎢</span></span></span><span style="top:-4.615960000000001em"><span class="pstrut" style="height:3.1550000000000002em"></span><span class="delimsizinginner delim-size4"><span>⎢</span></span></span><span style="top:-5.856980000000001em"><span class="pstrut" style="height:3.1550000000000002em"></span><span class="delimsizinginner delim-size4"><span>⎡</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:3.3500499999999995em"><span></span></span></span></span></span></span><span class="mord"><span class="mtable"><span class="col-align-c"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:3.850000000000001em"><span style="top:-6.010000000000001em"><span class="pstrut" style="height:3em"></span><span class="mord"><span class="mord"><span class="mord mathnormal">x</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:.30110799999999993em"><span style="top:-2.5500000000000003em;margin-left:0;margin-right:.05em"><span class="pstrut" style="height:2.7em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">0</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.15em"><span></span></span></span></span></span></span></span></span><span style="top:-4.810000000000001em"><span class="pstrut" style="height:3em"></span><span class="mord"><span class="mord"><span class="mord mathnormal">x</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:.30110799999999993em"><span style="top:-2.5500000000000003em;margin-left:0;margin-right:.05em"><span class="pstrut" style="height:2.7em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">1</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.15em"><span></span></span></span></span></span></span></span></span><span style="top:-3.6100000000000003em"><span class="pstrut" style="height:3em"></span><span class="mord"><span class="mord"><span class="mord mathnormal">x</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:.30110799999999993em"><span style="top:-2.5500000000000003em;margin-left:0;margin-right:.05em"><span class="pstrut" style="height:2.7em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">2</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.15em"><span></span></span></span></span></span></span></span></span><span style="top:-2.41em"><span class="pstrut" style="height:3em"></span><span class="mord"><span class="mord"><span class="mord mathnormal">x</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:.30110799999999993em"><span style="top:-2.5500000000000003em;margin-left:0;margin-right:.05em"><span class="pstrut" style="height:2.7em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">3</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.15em"><span></span></span></span></span></span></span></span></span><span style="top:-1.2100000000000002em"><span class="pstrut" style="height:3em"></span><span class="mord"><span class="minner">…</span></span></span><span style="top:-.009999999999999953em"><span class="pstrut" style="height:3em"></span><span class="mord"><span class="mord"><span class="mord mathnormal">x</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:.151392em"><span style="top:-2.5500000000000003em;margin-left:0;margin-right:.05em"><span class="pstrut" style="height:2.7em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight">n</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.15em"><span></span></span></span></span></span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:3.35em"><span></span></span></span></span></span></span></span><span class="mclose"><span class="delimsizing mult"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:3.8569800000000005em"><span style="top:-.44997000000000076em"><span class="pstrut" style="height:3.1550000000000002em"></span><span class="delimsizinginner delim-size4"><span>⎦</span></span></span><span style="top:-1.5999700000000008em"><span class="pstrut" style="height:3.1550000000000002em"></span><span class="delimsizinginner delim-size4"><span>⎥</span></span></span><span style="top:-2.195970000000001em"><span class="pstrut" style="height:3.1550000000000002em"></span><span class="delimsizinginner delim-size4"><span>⎥</span></span></span><span style="top:-2.791970000000001em"><span class="pstrut" style="height:3.1550000000000002em"></span><span class="delimsizinginner delim-size4"><span>⎥</span></span></span><span style="top:-3.3879700000000006em"><span class="pstrut" style="height:3.1550000000000002em"></span><span class="delimsizinginner delim-size4"><span>⎥</span></span></span><span style="top:-3.9839700000000007em"><span class="pstrut" style="height:3.1550000000000002em"></span><span class="delimsizinginner delim-size4"><span>⎥</span></span></span><span style="top:-4.579970000000001em"><span class="pstrut" style="height:3.1550000000000002em"></span><span class="delimsizinginner delim-size4"><span>⎥</span></span></span><span style="top:-4.615960000000001em"><span class="pstrut" style="height:3.1550000000000002em"></span><span class="delimsizinginner delim-size4"><span>⎥</span></span></span><span style="top:-5.856980000000001em"><span class="pstrut" style="height:3.1550000000000002em"></span><span class="delimsizinginner delim-size4"><span>⎤</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:3.3500499999999995em"><span></span></span></span></span></span></span></span><span class="mspace" style="margin-right:.16666666666666666em"></span><span class="mpunct">,</span><span class="mspace" style="margin-right:.16666666666666666em"></span><span class="mord mathnormal" style="margin-right:.02778em">θ</span><span class="mspace" style="margin-right:.2777777777777778em"></span><span class="mrel">=</span><span class="mspace" style="margin-right:.2777777777777778em"></span></span><span class="base"><span class="strut" style="height:7.20703em;vertical-align:-3.3500499999999995em"></span><span class="minner"><span class="mopen"><span class="delimsizing mult"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:3.8569800000000005em"><span style="top:-.44997000000000076em"><span class="pstrut" style="height:3.1550000000000002em"></span><span class="delimsizinginner delim-size4"><span>⎣</span></span></span><span style="top:-1.5999700000000008em"><span class="pstrut" style="height:3.1550000000000002em"></span><span class="delimsizinginner delim-size4"><span>⎢</span></span></span><span style="top:-2.195970000000001em"><span class="pstrut" style="height:3.1550000000000002em"></span><span class="delimsizinginner delim-size4"><span>⎢</span></span></span><span style="top:-2.791970000000001em"><span class="pstrut" style="height:3.1550000000000002em"></span><span class="delimsizinginner delim-size4"><span>⎢</span></span></span><span style="top:-3.3879700000000006em"><span class="pstrut" style="height:3.1550000000000002em"></span><span class="delimsizinginner delim-size4"><span>⎢</span></span></span><span style="top:-3.9839700000000007em"><span class="pstrut" style="height:3.1550000000000002em"></span><span class="delimsizinginner delim-size4"><span>⎢</span></span></span><span style="top:-4.579970000000001em"><span class="pstrut" style="height:3.1550000000000002em"></span><span class="delimsizinginner delim-size4"><span>⎢</span></span></span><span style="top:-4.615960000000001em"><span class="pstrut" style="height:3.1550000000000002em"></span><span class="delimsizinginner delim-size4"><span>⎢</span></span></span><span style="top:-5.856980000000001em"><span class="pstrut" style="height:3.1550000000000002em"></span><span class="delimsizinginner delim-size4"><span>⎡</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:3.3500499999999995em"><span></span></span></span></span></span></span><span class="mord"><span class="mtable"><span class="col-align-c"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:3.850000000000001em"><span style="top:-6.010000000000001em"><span class="pstrut" style="height:3em"></span><span class="mord"><span class="mord"><span class="mord mathnormal" style="margin-right:.02778em">θ</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:.30110799999999993em"><span style="top:-2.5500000000000003em;margin-left:-.02778em;margin-right:.05em"><span class="pstrut" style="height:2.7em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">0</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.15em"><span></span></span></span></span></span></span></span></span><span style="top:-4.810000000000001em"><span class="pstrut" style="height:3em"></span><span class="mord"><span class="mord"><span class="mord mathnormal" style="margin-right:.02778em">θ</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:.30110799999999993em"><span style="top:-2.5500000000000003em;margin-left:-.02778em;margin-right:.05em"><span class="pstrut" style="height:2.7em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">1</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.15em"><span></span></span></span></span></span></span></span></span><span style="top:-3.6100000000000003em"><span class="pstrut" style="height:3em"></span><span class="mord"><span class="mord"><span class="mord mathnormal" style="margin-right:.02778em">θ</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:.30110799999999993em"><span style="top:-2.5500000000000003em;margin-left:-.02778em;margin-right:.05em"><span class="pstrut" style="height:2.7em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">2</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.15em"><span></span></span></span></span></span></span></span></span><span style="top:-2.41em"><span class="pstrut" style="height:3em"></span><span class="mord"><span class="mord"><span class="mord mathnormal" style="margin-right:.02778em">θ</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:.30110799999999993em"><span style="top:-2.5500000000000003em;margin-left:-.02778em;margin-right:.05em"><span class="pstrut" style="height:2.7em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">3</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.15em"><span></span></span></span></span></span></span></span></span><span style="top:-1.2100000000000002em"><span class="pstrut" style="height:3em"></span><span class="mord"><span class="minner">…</span></span></span><span style="top:-.009999999999999953em"><span class="pstrut" style="height:3em"></span><span class="mord"><span class="mord"><span class="mord mathnormal" style="margin-right:.02778em">θ</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:.151392em"><span style="top:-2.5500000000000003em;margin-left:-.02778em;margin-right:.05em"><span class="pstrut" style="height:2.7em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight">n</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.15em"><span></span></span></span></span></span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:3.35em"><span></span></span></span></span></span></span></span><span class="mclose"><span class="delimsizing mult"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:3.8569800000000005em"><span style="top:-.44997000000000076em"><span class="pstrut" style="height:3.1550000000000002em"></span><span class="delimsizinginner delim-size4"><span>⎦</span></span></span><span style="top:-1.5999700000000008em"><span class="pstrut" style="height:3.1550000000000002em"></span><span class="delimsizinginner delim-size4"><span>⎥</span></span></span><span style="top:-2.195970000000001em"><span class="pstrut" style="height:3.1550000000000002em"></span><span class="delimsizinginner delim-size4"><span>⎥</span></span></span><span style="top:-2.791970000000001em"><span class="pstrut" style="height:3.1550000000000002em"></span><span class="delimsizinginner delim-size4"><span>⎥</span></span></span><span style="top:-3.3879700000000006em"><span class="pstrut" style="height:3.1550000000000002em"></span><span class="delimsizinginner delim-size4"><span>⎥</span></span></span><span style="top:-3.9839700000000007em"><span class="pstrut" style="height:3.1550000000000002em"></span><span class="delimsizinginner delim-size4"><span>⎥</span></span></span><span style="top:-4.579970000000001em"><span class="pstrut" style="height:3.1550000000000002em"></span><span class="delimsizinginner delim-size4"><span>⎥</span></span></span><span style="top:-4.615960000000001em"><span class="pstrut" style="height:3.1550000000000002em"></span><span class="delimsizinginner delim-size4"><span>⎥</span></span></span><span style="top:-5.856980000000001em"><span class="pstrut" style="height:3.1550000000000002em"></span><span class="delimsizinginner delim-size4"><span>⎤</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:3.3500499999999995em"><span></span></span></span></span></span></span></span></span></span></span></span></p><p>这样方程就变为：</p><p><span class="katex-display"><span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML" display="block"><semantics><mrow><msub><mi>h</mi><mi>θ</mi></msub><mo stretchy="false">(</mo><mi>x</mi><mo stretchy="false">)</mo><mo>=</mo><msup><mrow><mo fence="true">[</mo><mtable rowspacing="0.15999999999999992em" columnspacing="1em"><mtr><mtd><mstyle scriptlevel="0" displaystyle="false"><msub><mi>θ</mi><mn>0</mn></msub></mstyle></mtd><mtd><mstyle scriptlevel="0" displaystyle="false"><msub><mi>θ</mi><mn>1</mn></msub></mstyle></mtd><mtd><mstyle scriptlevel="0" displaystyle="false"><msub><mi>θ</mi><mn>2</mn></msub></mstyle></mtd><mtd><mstyle scriptlevel="0" displaystyle="false"><msub><mi>θ</mi><mn>3</mn></msub></mstyle></mtd><mtd><mstyle scriptlevel="0" displaystyle="false"><mo lspace="0em" rspace="0em">…</mo></mstyle></mtd><mtd><mstyle scriptlevel="0" displaystyle="false"><msub><mi>θ</mi><mi>n</mi></msub></mstyle></mtd></mtr></mtable><mo fence="true">]</mo></mrow><mi>T</mi></msup><mrow><mo fence="true">[</mo><mtable rowspacing="0.15999999999999992em" columnspacing="1em"><mtr><mtd><mstyle scriptlevel="0" displaystyle="false"><msub><mi>x</mi><mn>0</mn></msub></mstyle></mtd></mtr><mtr><mtd><mstyle scriptlevel="0" displaystyle="false"><msub><mi>x</mi><mn>1</mn></msub></mstyle></mtd></mtr><mtr><mtd><mstyle scriptlevel="0" displaystyle="false"><msub><mi>x</mi><mn>2</mn></msub></mstyle></mtd></mtr><mtr><mtd><mstyle scriptlevel="0" displaystyle="false"><msub><mi>x</mi><mn>3</mn></msub></mstyle></mtd></mtr><mtr><mtd><mstyle scriptlevel="0" displaystyle="false"><mo lspace="0em" rspace="0em">…</mo></mstyle></mtd></mtr><mtr><mtd><mstyle scriptlevel="0" displaystyle="false"><msub><mi>x</mi><mi>n</mi></msub></mstyle></mtd></mtr></mtable><mo fence="true">]</mo></mrow></mrow><annotation encoding="application/x-tex">h_\theta(x)=\begin{bmatrix} \theta_0 &amp; \theta_1 &amp; \theta_2 &amp; \theta_3 &amp; … &amp; \theta_n\end{bmatrix}^T\begin{bmatrix} x_0 \\ x_1 \\ x_2 \\ x_3 \\ … \\ x_n \end{bmatrix}</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:1em;vertical-align:-.25em"></span><span class="mord"><span class="mord mathnormal">h</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:.33610799999999996em"><span style="top:-2.5500000000000003em;margin-left:0;margin-right:.05em"><span class="pstrut" style="height:2.7em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight" style="margin-right:.02778em">θ</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.15em"><span></span></span></span></span></span></span><span class="mopen">(</span><span class="mord mathnormal">x</span><span class="mclose">)</span><span class="mspace" style="margin-right:.2777777777777778em"></span><span class="mrel">=</span><span class="mspace" style="margin-right:.2777777777777778em"></span></span><span class="base"><span class="strut" style="height:7.20703em;vertical-align:-3.3500499999999995em"></span><span class="minner"><span class="minner"><span class="mopen delimcenter" style="top:0"><span class="delimsizing size1">[</span></span><span class="mord"><span class="mtable"><span class="col-align-c"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:.8500000000000001em"><span style="top:-3.01em"><span class="pstrut" style="height:3em"></span><span class="mord"><span class="mord"><span class="mord mathnormal" style="margin-right:.02778em">θ</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:.30110799999999993em"><span style="top:-2.5500000000000003em;margin-left:-.02778em;margin-right:.05em"><span class="pstrut" style="height:2.7em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">0</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.15em"><span></span></span></span></span></span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.35000000000000003em"><span></span></span></span></span></span><span class="arraycolsep" style="width:.5em"></span><span class="arraycolsep" style="width:.5em"></span><span class="col-align-c"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:.8500000000000001em"><span style="top:-3.01em"><span class="pstrut" style="height:3em"></span><span class="mord"><span class="mord"><span class="mord mathnormal" style="margin-right:.02778em">θ</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:.30110799999999993em"><span style="top:-2.5500000000000003em;margin-left:-.02778em;margin-right:.05em"><span class="pstrut" style="height:2.7em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">1</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.15em"><span></span></span></span></span></span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.35000000000000003em"><span></span></span></span></span></span><span class="arraycolsep" style="width:.5em"></span><span class="arraycolsep" style="width:.5em"></span><span class="col-align-c"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:.8500000000000001em"><span style="top:-3.01em"><span class="pstrut" style="height:3em"></span><span class="mord"><span class="mord"><span class="mord mathnormal" style="margin-right:.02778em">θ</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:.30110799999999993em"><span style="top:-2.5500000000000003em;margin-left:-.02778em;margin-right:.05em"><span class="pstrut" style="height:2.7em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">2</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.15em"><span></span></span></span></span></span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.35000000000000003em"><span></span></span></span></span></span><span class="arraycolsep" style="width:.5em"></span><span class="arraycolsep" style="width:.5em"></span><span class="col-align-c"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:.8500000000000001em"><span style="top:-3.01em"><span class="pstrut" style="height:3em"></span><span class="mord"><span class="mord"><span class="mord mathnormal" style="margin-right:.02778em">θ</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:.30110799999999993em"><span style="top:-2.5500000000000003em;margin-left:-.02778em;margin-right:.05em"><span class="pstrut" style="height:2.7em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">3</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.15em"><span></span></span></span></span></span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.35000000000000003em"><span></span></span></span></span></span><span class="arraycolsep" style="width:.5em"></span><span class="arraycolsep" style="width:.5em"></span><span class="col-align-c"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:.8500000000000001em"><span style="top:-3.01em"><span class="pstrut" style="height:3em"></span><span class="mord"><span class="minner">…</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.35000000000000003em"><span></span></span></span></span></span><span class="arraycolsep" style="width:.5em"></span><span class="arraycolsep" style="width:.5em"></span><span class="col-align-c"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:.8500000000000001em"><span style="top:-3.01em"><span class="pstrut" style="height:3em"></span><span class="mord"><span class="mord"><span class="mord mathnormal" style="margin-right:.02778em">θ</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:.151392em"><span style="top:-2.5500000000000003em;margin-left:-.02778em;margin-right:.05em"><span class="pstrut" style="height:2.7em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight">n</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.15em"><span></span></span></span></span></span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.35000000000000003em"><span></span></span></span></span></span></span></span><span class="mclose delimcenter" style="top:0"><span class="delimsizing size1">]</span></span></span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height:1.081231em"><span style="top:-3.3029em;margin-right:.05em"><span class="pstrut" style="height:2.7em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight" style="margin-right:.13889em">T</span></span></span></span></span></span></span></span><span class="mspace" style="margin-right:.16666666666666666em"></span><span class="minner"><span class="mopen"><span class="delimsizing mult"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:3.8569800000000005em"><span style="top:-.44997000000000076em"><span class="pstrut" style="height:3.1550000000000002em"></span><span class="delimsizinginner delim-size4"><span>⎣</span></span></span><span style="top:-1.5999700000000008em"><span class="pstrut" style="height:3.1550000000000002em"></span><span class="delimsizinginner delim-size4"><span>⎢</span></span></span><span style="top:-2.195970000000001em"><span class="pstrut" style="height:3.1550000000000002em"></span><span class="delimsizinginner delim-size4"><span>⎢</span></span></span><span style="top:-2.791970000000001em"><span class="pstrut" style="height:3.1550000000000002em"></span><span class="delimsizinginner delim-size4"><span>⎢</span></span></span><span style="top:-3.3879700000000006em"><span class="pstrut" style="height:3.1550000000000002em"></span><span class="delimsizinginner delim-size4"><span>⎢</span></span></span><span style="top:-3.9839700000000007em"><span class="pstrut" style="height:3.1550000000000002em"></span><span class="delimsizinginner delim-size4"><span>⎢</span></span></span><span style="top:-4.579970000000001em"><span class="pstrut" style="height:3.1550000000000002em"></span><span class="delimsizinginner delim-size4"><span>⎢</span></span></span><span style="top:-4.615960000000001em"><span class="pstrut" style="height:3.1550000000000002em"></span><span class="delimsizinginner delim-size4"><span>⎢</span></span></span><span style="top:-5.856980000000001em"><span class="pstrut" style="height:3.1550000000000002em"></span><span class="delimsizinginner delim-size4"><span>⎡</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:3.3500499999999995em"><span></span></span></span></span></span></span><span class="mord"><span class="mtable"><span class="col-align-c"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:3.850000000000001em"><span style="top:-6.010000000000001em"><span class="pstrut" style="height:3em"></span><span class="mord"><span class="mord"><span class="mord mathnormal">x</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:.30110799999999993em"><span style="top:-2.5500000000000003em;margin-left:0;margin-right:.05em"><span class="pstrut" style="height:2.7em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">0</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.15em"><span></span></span></span></span></span></span></span></span><span style="top:-4.810000000000001em"><span class="pstrut" style="height:3em"></span><span class="mord"><span class="mord"><span class="mord mathnormal">x</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:.30110799999999993em"><span style="top:-2.5500000000000003em;margin-left:0;margin-right:.05em"><span class="pstrut" style="height:2.7em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">1</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.15em"><span></span></span></span></span></span></span></span></span><span style="top:-3.6100000000000003em"><span class="pstrut" style="height:3em"></span><span class="mord"><span class="mord"><span class="mord mathnormal">x</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:.30110799999999993em"><span style="top:-2.5500000000000003em;margin-left:0;margin-right:.05em"><span class="pstrut" style="height:2.7em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">2</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.15em"><span></span></span></span></span></span></span></span></span><span style="top:-2.41em"><span class="pstrut" style="height:3em"></span><span class="mord"><span class="mord"><span class="mord mathnormal">x</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:.30110799999999993em"><span style="top:-2.5500000000000003em;margin-left:0;margin-right:.05em"><span class="pstrut" style="height:2.7em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">3</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.15em"><span></span></span></span></span></span></span></span></span><span style="top:-1.2100000000000002em"><span class="pstrut" style="height:3em"></span><span class="mord"><span class="minner">…</span></span></span><span style="top:-.009999999999999953em"><span class="pstrut" style="height:3em"></span><span class="mord"><span class="mord"><span class="mord mathnormal">x</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:.151392em"><span style="top:-2.5500000000000003em;margin-left:0;margin-right:.05em"><span class="pstrut" style="height:2.7em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight">n</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.15em"><span></span></span></span></span></span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:3.35em"><span></span></span></span></span></span></span></span><span class="mclose"><span class="delimsizing mult"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:3.8569800000000005em"><span style="top:-.44997000000000076em"><span class="pstrut" style="height:3.1550000000000002em"></span><span class="delimsizinginner delim-size4"><span>⎦</span></span></span><span style="top:-1.5999700000000008em"><span class="pstrut" style="height:3.1550000000000002em"></span><span class="delimsizinginner delim-size4"><span>⎥</span></span></span><span style="top:-2.195970000000001em"><span class="pstrut" style="height:3.1550000000000002em"></span><span class="delimsizinginner delim-size4"><span>⎥</span></span></span><span style="top:-2.791970000000001em"><span class="pstrut" style="height:3.1550000000000002em"></span><span class="delimsizinginner delim-size4"><span>⎥</span></span></span><span style="top:-3.3879700000000006em"><span class="pstrut" style="height:3.1550000000000002em"></span><span class="delimsizinginner delim-size4"><span>⎥</span></span></span><span style="top:-3.9839700000000007em"><span class="pstrut" style="height:3.1550000000000002em"></span><span class="delimsizinginner delim-size4"><span>⎥</span></span></span><span style="top:-4.579970000000001em"><span class="pstrut" style="height:3.1550000000000002em"></span><span class="delimsizinginner delim-size4"><span>⎥</span></span></span><span style="top:-4.615960000000001em"><span class="pstrut" style="height:3.1550000000000002em"></span><span class="delimsizinginner delim-size4"><span>⎥</span></span></span><span style="top:-5.856980000000001em"><span class="pstrut" style="height:3.1550000000000002em"></span><span class="delimsizinginner delim-size4"><span>⎤</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:3.3500499999999995em"><span></span></span></span></span></span></span></span></span></span></span></span></p><p>由回归方程推导出损失方程：</p><p><img data-src="https://shun309.oss-cn-hangzhou.aliyuncs.com/photos/20210415231826.png" alt=""></p><h2 id="43-梯度下降"><a class="anchor" href="#43-梯度下降">#</a> 4.3 梯度下降</h2><p><img data-src="https://shun309.oss-cn-hangzhou.aliyuncs.com/photos/20210415232202.png" alt=""></p><p>计算时要从<span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><msub><mi>θ</mi><mn>0</mn></msub></mrow><annotation encoding="application/x-tex">\theta_0</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:.84444em;vertical-align:-.15em"></span><span class="mord"><span class="mord mathnormal" style="margin-right:.02778em">θ</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:.30110799999999993em"><span style="top:-2.5500000000000003em;margin-left:-.02778em;margin-right:.05em"><span class="pstrut" style="height:2.7em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">0</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.15em"><span></span></span></span></span></span></span></span></span></span> 一直计算到<span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><msub><mi>θ</mi><mi>n</mi></msub></mrow><annotation encoding="application/x-tex">\theta_n</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:.84444em;vertical-align:-.15em"></span><span class="mord"><span class="mord mathnormal" style="margin-right:.02778em">θ</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:.151392em"><span style="top:-2.5500000000000003em;margin-left:-.02778em;margin-right:.05em"><span class="pstrut" style="height:2.7em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight">n</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.15em"><span></span></span></span></span></span></span></span></span></span> 后，再从头由<span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><msub><mi>θ</mi><mn>0</mn></msub></mrow><annotation encoding="application/x-tex">\theta_0</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:.84444em;vertical-align:-.15em"></span><span class="mord"><span class="mord mathnormal" style="margin-right:.02778em">θ</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:.30110799999999993em"><span style="top:-2.5500000000000003em;margin-left:-.02778em;margin-right:.05em"><span class="pstrut" style="height:2.7em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">0</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.15em"><span></span></span></span></span></span></span></span></span></span> 开始计算，以确保数据统一变化。</p><p>在执行次数足够多的迭代后，我们就能取得达到要求的<span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><msub><mi>θ</mi><mi>j</mi></msub></mrow><annotation encoding="application/x-tex">\theta_j</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:.980548em;vertical-align:-.286108em"></span><span class="mord"><span class="mord mathnormal" style="margin-right:.02778em">θ</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:.311664em"><span style="top:-2.5500000000000003em;margin-left:-.02778em;margin-right:.05em"><span class="pstrut" style="height:2.7em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight" style="margin-right:.05724em">j</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:.286108em"><span></span></span></span></span></span></span></span></span></span> 的值。</p><h1 id="5-鸢尾花数据集"><a class="anchor" href="#5-鸢尾花数据集">#</a> 5 鸢尾花数据集</h1><p>Iris 鸢尾花数据集内包含 3 类，分别为<strong>山鸢尾（Iris-setosa）</strong>、<strong>变色鸢尾（Iris-versicolor）<strong>和</strong>维吉尼亚鸢尾（Iris-virginica）</strong>，共 150 条记录，每类各 50 个数据，每条记录都有 4 项特征：花萼长度、花萼宽度、花瓣长度、花瓣宽度，可以通过这 4 个特征预测鸢尾花卉属于哪一品种。 这是本文章所使用的鸢尾花数据集： <strong>sl：花萼长度 ；sw：花萼宽度 ；pl：花瓣长度 ；pw：花瓣宽度； type：类别：（Iris-setosa、Iris-versicolor、Iris-virginica 三类）</strong></p><p><img data-src="https://jiang-hs.github.io/post-images/1594871083036.jpg" alt=""></p><p><strong>鸢尾花数据集下载</strong>:<br><span class="exturl" data-url="aHR0cHM6Ly9hcmNoaXZlLmljcy51Y2kuZWR1L21sL21hY2hpbmUtbGVhcm5pbmctZGF0YWJhc2VzL2lyaXMv">https://archive.ics.uci.edu/ml/machine-learning-databases/iris/</span></p><p><img data-src="https://shun309.oss-cn-hangzhou.aliyuncs.com/photos/20210415234133.png" alt=""></p><p>下载这个 <code>iris.data</code> 即可<br>将其置于当前工作文件夹即可</p><hr><p>先导入需要的库:</p><figure class="highlight python"><figcaption data-lang="python"></figcaption><table><tr><td data-num="1"></td><td><pre><span class="token keyword">import</span> numpy <span class="token keyword">as</span> np</pre></td></tr><tr><td data-num="2"></td><td><pre><span class="token keyword">import</span> pandas <span class="token keyword">as</span> pd</pre></td></tr><tr><td data-num="3"></td><td><pre><span class="token keyword">import</span> random</pre></td></tr><tr><td data-num="4"></td><td><pre><span class="token keyword">import</span> time</pre></td></tr></table></figure><p>定义模型：</p><figure class="highlight python"><figcaption data-lang="python"></figcaption><table><tr><td data-num="1"></td><td><pre><span class="token keyword">def</span> <span class="token function">MGD_train</span><span class="token punctuation">(</span>X<span class="token punctuation">,</span> y<span class="token punctuation">,</span> alpha<span class="token operator">=</span><span class="token number">0.0001</span><span class="token punctuation">,</span> maxIter<span class="token operator">=</span><span class="token number">1000</span><span class="token punctuation">,</span> theta_old<span class="token operator">=</span><span class="token boolean">None</span><span class="token punctuation">)</span><span class="token punctuation">:</span></pre></td></tr><tr><td data-num="2"></td><td><pre>    <span class="token triple-quoted-string string">'''</span></pre></td></tr><tr><td data-num="3"></td><td><pre>    MGD训练线性回归</pre></td></tr><tr><td data-num="4"></td><td><pre>    传入:</pre></td></tr><tr><td data-num="5"></td><td><pre>        X       :  已知数据  </pre></td></tr><tr><td data-num="6"></td><td><pre>        y       :  标签</pre></td></tr><tr><td data-num="7"></td><td><pre>        alpha   :  学习率</pre></td></tr><tr><td data-num="8"></td><td><pre>        maxIter :  总迭代次数 </pre></td></tr><tr><td data-num="9"></td><td><pre>    返回:</pre></td></tr><tr><td data-num="10"></td><td><pre>        theta : 权重参数</pre></td></tr><tr><td data-num="11"></td><td><pre>    '''</pre></td></tr><tr><td data-num="12"></td><td><pre>    <span class="token comment"># 初始化权重参数</span></pre></td></tr><tr><td data-num="13"></td><td><pre>    theta <span class="token operator">=</span> np<span class="token punctuation">.</span>ones<span class="token punctuation">(</span>shape<span class="token operator">=</span><span class="token punctuation">(</span>X<span class="token punctuation">.</span>shape<span class="token punctuation">[</span><span class="token number">1</span><span class="token punctuation">]</span><span class="token punctuation">,</span><span class="token punctuation">)</span><span class="token punctuation">)</span></pre></td></tr><tr><td data-num="14"></td><td><pre>    <span class="token keyword">if</span> <span class="token keyword">not</span> theta_old <span class="token keyword">is</span> <span class="token boolean">None</span><span class="token punctuation">:</span></pre></td></tr><tr><td data-num="15"></td><td><pre>        <span class="token comment"># 假装是断点续训练</span></pre></td></tr><tr><td data-num="16"></td><td><pre>        theta <span class="token operator">=</span> theta_old<span class="token punctuation">.</span>copy<span class="token punctuation">(</span><span class="token punctuation">)</span></pre></td></tr><tr><td data-num="17"></td><td><pre>    </pre></td></tr><tr><td data-num="18"></td><td><pre>    <span class="token comment">#axis=1 表示横轴，方向从左到右；axis=0 表示纵轴，方向从上到下</span></pre></td></tr><tr><td data-num="19"></td><td><pre>    <span class="token keyword">for</span> i <span class="token keyword">in</span> <span class="token builtin">range</span><span class="token punctuation">(</span>maxIter<span class="token punctuation">)</span><span class="token punctuation">:</span></pre></td></tr><tr><td data-num="20"></td><td><pre>        <span class="token comment"># 预测</span></pre></td></tr><tr><td data-num="21"></td><td><pre>        y_pred <span class="token operator">=</span> np<span class="token punctuation">.</span><span class="token builtin">sum</span><span class="token punctuation">(</span>X <span class="token operator">*</span> theta<span class="token punctuation">,</span> axis<span class="token operator">=</span><span class="token number">1</span><span class="token punctuation">)</span></pre></td></tr><tr><td data-num="22"></td><td><pre>        <span class="token comment"># 全部数据得到的梯度</span></pre></td></tr><tr><td data-num="23"></td><td><pre>        gradient <span class="token operator">=</span> np<span class="token punctuation">.</span>average<span class="token punctuation">(</span><span class="token punctuation">(</span>y <span class="token operator">-</span> y_pred<span class="token punctuation">)</span><span class="token punctuation">.</span>reshape<span class="token punctuation">(</span><span class="token operator">-</span><span class="token number">1</span><span class="token punctuation">,</span> <span class="token number">1</span><span class="token punctuation">)</span> <span class="token operator">*</span> X<span class="token punctuation">,</span> axis<span class="token operator">=</span><span class="token number">0</span><span class="token punctuation">)</span></pre></td></tr><tr><td data-num="24"></td><td><pre>        <span class="token comment"># 更新学习率</span></pre></td></tr><tr><td data-num="25"></td><td><pre>        theta <span class="token operator">+=</span> alpha <span class="token operator">*</span> gradient</pre></td></tr><tr><td data-num="26"></td><td><pre>    <span class="token keyword">return</span> theta</pre></td></tr><tr><td data-num="27"></td><td><pre></pre></td></tr><tr><td data-num="28"></td><td><pre></pre></td></tr><tr><td data-num="29"></td><td><pre><span class="token keyword">def</span> <span class="token function">SGD_train</span><span class="token punctuation">(</span>X<span class="token punctuation">,</span> y<span class="token punctuation">,</span> alpha<span class="token operator">=</span><span class="token number">0.0001</span><span class="token punctuation">,</span> maxIter<span class="token operator">=</span><span class="token number">1000</span><span class="token punctuation">,</span> theta_old<span class="token operator">=</span><span class="token boolean">None</span><span class="token punctuation">)</span><span class="token punctuation">:</span></pre></td></tr><tr><td data-num="30"></td><td><pre>    <span class="token triple-quoted-string string">'''</span></pre></td></tr><tr><td data-num="31"></td><td><pre>    SGD训练线性回归</pre></td></tr><tr><td data-num="32"></td><td><pre>    传入:</pre></td></tr><tr><td data-num="33"></td><td><pre>        X       :  已知数据  </pre></td></tr><tr><td data-num="34"></td><td><pre>        y       :  标签</pre></td></tr><tr><td data-num="35"></td><td><pre>        alpha   :  学习率</pre></td></tr><tr><td data-num="36"></td><td><pre>        maxIter :  总迭代次数</pre></td></tr><tr><td data-num="37"></td><td><pre>        </pre></td></tr><tr><td data-num="38"></td><td><pre>    返回:</pre></td></tr><tr><td data-num="39"></td><td><pre>        theta : 权重参数</pre></td></tr><tr><td data-num="40"></td><td><pre>    '''</pre></td></tr><tr><td data-num="41"></td><td><pre>    <span class="token comment"># 初始化权重参数</span></pre></td></tr><tr><td data-num="42"></td><td><pre>    theta <span class="token operator">=</span> np<span class="token punctuation">.</span>ones<span class="token punctuation">(</span>shape<span class="token operator">=</span><span class="token punctuation">(</span>X<span class="token punctuation">.</span>shape<span class="token punctuation">[</span><span class="token number">1</span><span class="token punctuation">]</span><span class="token punctuation">,</span><span class="token punctuation">)</span><span class="token punctuation">)</span></pre></td></tr><tr><td data-num="43"></td><td><pre>    <span class="token keyword">if</span> <span class="token keyword">not</span> theta_old <span class="token keyword">is</span> <span class="token boolean">None</span><span class="token punctuation">:</span></pre></td></tr><tr><td data-num="44"></td><td><pre>        <span class="token comment"># 假装是断点续训练</span></pre></td></tr><tr><td data-num="45"></td><td><pre>        theta <span class="token operator">=</span> theta_old<span class="token punctuation">.</span>copy<span class="token punctuation">(</span><span class="token punctuation">)</span></pre></td></tr><tr><td data-num="46"></td><td><pre>    <span class="token comment"># 数据数量</span></pre></td></tr><tr><td data-num="47"></td><td><pre>    data_length <span class="token operator">=</span> X<span class="token punctuation">.</span>shape<span class="token punctuation">[</span><span class="token number">0</span><span class="token punctuation">]</span></pre></td></tr><tr><td data-num="48"></td><td><pre>    <span class="token keyword">for</span> i <span class="token keyword">in</span> <span class="token builtin">range</span><span class="token punctuation">(</span>maxIter<span class="token punctuation">)</span><span class="token punctuation">:</span></pre></td></tr><tr><td data-num="49"></td><td><pre>        <span class="token comment"># 随机选择一个数据</span></pre></td></tr><tr><td data-num="50"></td><td><pre>        index <span class="token operator">=</span> np<span class="token punctuation">.</span>random<span class="token punctuation">.</span>randint<span class="token punctuation">(</span><span class="token number">0</span><span class="token punctuation">,</span> data_length<span class="token punctuation">)</span></pre></td></tr><tr><td data-num="51"></td><td><pre>        <span class="token comment"># 预测</span></pre></td></tr><tr><td data-num="52"></td><td><pre>        y_pred <span class="token operator">=</span> np<span class="token punctuation">.</span><span class="token builtin">sum</span><span class="token punctuation">(</span>X<span class="token punctuation">[</span>index<span class="token punctuation">,</span> <span class="token punctuation">:</span><span class="token punctuation">]</span> <span class="token operator">*</span> theta<span class="token punctuation">)</span></pre></td></tr><tr><td data-num="53"></td><td><pre>        <span class="token comment"># 一条数据得到的梯度</span></pre></td></tr><tr><td data-num="54"></td><td><pre>        gradient <span class="token operator">=</span> <span class="token punctuation">(</span>y<span class="token punctuation">[</span>index<span class="token punctuation">]</span> <span class="token operator">-</span> y_pred<span class="token punctuation">)</span> <span class="token operator">*</span> X<span class="token punctuation">[</span>index<span class="token punctuation">,</span> <span class="token punctuation">:</span><span class="token punctuation">]</span></pre></td></tr><tr><td data-num="55"></td><td><pre>        <span class="token comment"># 更新学习率</span></pre></td></tr><tr><td data-num="56"></td><td><pre>        theta <span class="token operator">+=</span> alpha <span class="token operator">*</span> gradient</pre></td></tr><tr><td data-num="57"></td><td><pre>    <span class="token keyword">return</span> theta</pre></td></tr><tr><td data-num="58"></td><td><pre>    </pre></td></tr><tr><td data-num="59"></td><td><pre>    </pre></td></tr><tr><td data-num="60"></td><td><pre><span class="token keyword">def</span> <span class="token function">MBGD_train</span><span class="token punctuation">(</span>X<span class="token punctuation">,</span> y<span class="token punctuation">,</span> alpha<span class="token operator">=</span><span class="token number">0.0001</span><span class="token punctuation">,</span> maxIter<span class="token operator">=</span><span class="token number">1000</span><span class="token punctuation">,</span> batch_size<span class="token operator">=</span><span class="token number">10</span><span class="token punctuation">,</span> theta_old<span class="token operator">=</span><span class="token boolean">None</span><span class="token punctuation">)</span><span class="token punctuation">:</span></pre></td></tr><tr><td data-num="61"></td><td><pre>    <span class="token triple-quoted-string string">'''</span></pre></td></tr><tr><td data-num="62"></td><td><pre>    MBGD训练线性回归</pre></td></tr><tr><td data-num="63"></td><td><pre>    传入:</pre></td></tr><tr><td data-num="64"></td><td><pre>        X          :  已知数据  </pre></td></tr><tr><td data-num="65"></td><td><pre>        y          :  标签</pre></td></tr><tr><td data-num="66"></td><td><pre>        alpha      :  学习率</pre></td></tr><tr><td data-num="67"></td><td><pre>        maxIter    :  总迭代次数</pre></td></tr><tr><td data-num="68"></td><td><pre>        batch_size :  没一轮喂入的数据数</pre></td></tr><tr><td data-num="69"></td><td><pre>        </pre></td></tr><tr><td data-num="70"></td><td><pre>    返回:</pre></td></tr><tr><td data-num="71"></td><td><pre>        theta : 权重参数</pre></td></tr><tr><td data-num="72"></td><td><pre>    '''</pre></td></tr><tr><td data-num="73"></td><td><pre>    <span class="token comment"># 初始化权重参数</span></pre></td></tr><tr><td data-num="74"></td><td><pre>    theta <span class="token operator">=</span> np<span class="token punctuation">.</span>ones<span class="token punctuation">(</span>shape<span class="token operator">=</span><span class="token punctuation">(</span>X<span class="token punctuation">.</span>shape<span class="token punctuation">[</span><span class="token number">1</span><span class="token punctuation">]</span><span class="token punctuation">,</span><span class="token punctuation">)</span><span class="token punctuation">)</span></pre></td></tr><tr><td data-num="75"></td><td><pre>    </pre></td></tr><tr><td data-num="76"></td><td><pre>    <span class="token keyword">if</span> <span class="token keyword">not</span> theta_old <span class="token keyword">is</span> <span class="token boolean">None</span><span class="token punctuation">:</span></pre></td></tr><tr><td data-num="77"></td><td><pre>        <span class="token comment"># 假装是断点续训练</span></pre></td></tr><tr><td data-num="78"></td><td><pre>        theta <span class="token operator">=</span> theta_old<span class="token punctuation">.</span>copy<span class="token punctuation">(</span><span class="token punctuation">)</span></pre></td></tr><tr><td data-num="79"></td><td><pre>    </pre></td></tr><tr><td data-num="80"></td><td><pre>    <span class="token comment"># 所有数据的集合</span></pre></td></tr><tr><td data-num="81"></td><td><pre>    all_data <span class="token operator">=</span> np<span class="token punctuation">.</span>concatenate<span class="token punctuation">(</span><span class="token punctuation">[</span>X<span class="token punctuation">,</span> y<span class="token punctuation">.</span>reshape<span class="token punctuation">(</span><span class="token operator">-</span><span class="token number">1</span><span class="token punctuation">,</span> <span class="token number">1</span><span class="token punctuation">)</span><span class="token punctuation">]</span><span class="token punctuation">,</span> axis<span class="token operator">=</span><span class="token number">1</span><span class="token punctuation">)</span></pre></td></tr><tr><td data-num="82"></td><td><pre>    <span class="token keyword">for</span> i <span class="token keyword">in</span> <span class="token builtin">range</span><span class="token punctuation">(</span>maxIter<span class="token punctuation">)</span><span class="token punctuation">:</span></pre></td></tr><tr><td data-num="83"></td><td><pre>        <span class="token comment"># 从全部数据里选 batch_size 个 item</span></pre></td></tr><tr><td data-num="84"></td><td><pre>        X_batch_size <span class="token operator">=</span> np<span class="token punctuation">.</span>array<span class="token punctuation">(</span>random<span class="token punctuation">.</span>choices<span class="token punctuation">(</span>all_data<span class="token punctuation">,</span> k<span class="token operator">=</span>batch_size<span class="token punctuation">)</span><span class="token punctuation">)</span></pre></td></tr><tr><td data-num="85"></td><td><pre>        </pre></td></tr><tr><td data-num="86"></td><td><pre>        <span class="token comment"># 重新给 X, y 赋值</span></pre></td></tr><tr><td data-num="87"></td><td><pre>        X_new <span class="token operator">=</span> X_batch_size<span class="token punctuation">[</span><span class="token punctuation">:</span><span class="token punctuation">,</span> <span class="token punctuation">:</span><span class="token operator">-</span><span class="token number">1</span><span class="token punctuation">]</span></pre></td></tr><tr><td data-num="88"></td><td><pre>        y_new <span class="token operator">=</span> X_batch_size<span class="token punctuation">[</span><span class="token punctuation">:</span><span class="token punctuation">,</span> <span class="token operator">-</span><span class="token number">1</span><span class="token punctuation">]</span></pre></td></tr><tr><td data-num="89"></td><td><pre>        </pre></td></tr><tr><td data-num="90"></td><td><pre>        <span class="token comment"># 将数据喂入，更新 theta</span></pre></td></tr><tr><td data-num="91"></td><td><pre>        theta <span class="token operator">=</span> MGD_train<span class="token punctuation">(</span>X_new<span class="token punctuation">,</span> y_new<span class="token punctuation">,</span> alpha<span class="token operator">=</span><span class="token number">0.0001</span><span class="token punctuation">,</span> maxIter<span class="token operator">=</span><span class="token number">1</span><span class="token punctuation">,</span> theta_old<span class="token operator">=</span>theta<span class="token punctuation">)</span></pre></td></tr><tr><td data-num="92"></td><td><pre>    <span class="token keyword">return</span> theta</pre></td></tr><tr><td data-num="93"></td><td><pre></pre></td></tr><tr><td data-num="94"></td><td><pre></pre></td></tr><tr><td data-num="95"></td><td><pre><span class="token keyword">def</span> <span class="token function">GD_predict</span><span class="token punctuation">(</span>X<span class="token punctuation">,</span> theta<span class="token punctuation">)</span><span class="token punctuation">:</span></pre></td></tr><tr><td data-num="96"></td><td><pre>    <span class="token triple-quoted-string string">'''</span></pre></td></tr><tr><td data-num="97"></td><td><pre>    用于预测的函数</pre></td></tr><tr><td data-num="98"></td><td><pre>    传入:</pre></td></tr><tr><td data-num="99"></td><td><pre>        X     : 数据</pre></td></tr><tr><td data-num="100"></td><td><pre>        theta : 权重</pre></td></tr><tr><td data-num="101"></td><td><pre>    返回:</pre></td></tr><tr><td data-num="102"></td><td><pre>        y_pred: 预测向量</pre></td></tr><tr><td data-num="103"></td><td><pre>    '''</pre></td></tr><tr><td data-num="104"></td><td><pre>    y_pred <span class="token operator">=</span> np<span class="token punctuation">.</span><span class="token builtin">sum</span><span class="token punctuation">(</span>theta <span class="token operator">*</span> X<span class="token punctuation">,</span> axis<span class="token operator">=</span><span class="token number">1</span><span class="token punctuation">)</span></pre></td></tr><tr><td data-num="105"></td><td><pre>    <span class="token comment"># 实数域空间 -> 离散三值空间，则需要四舍五入</span></pre></td></tr><tr><td data-num="106"></td><td><pre>    y_pred <span class="token operator">=</span> <span class="token punctuation">(</span>y_pred <span class="token operator">+</span> <span class="token number">0.5</span><span class="token punctuation">)</span><span class="token punctuation">.</span>astype<span class="token punctuation">(</span><span class="token builtin">int</span><span class="token punctuation">)</span></pre></td></tr><tr><td data-num="107"></td><td><pre>    <span class="token keyword">return</span> y_pred </pre></td></tr><tr><td data-num="108"></td><td><pre></pre></td></tr><tr><td data-num="109"></td><td><pre></pre></td></tr><tr><td data-num="110"></td><td><pre><span class="token keyword">def</span> <span class="token function">calc_accuracy</span><span class="token punctuation">(</span>y<span class="token punctuation">,</span> y_pred<span class="token punctuation">)</span><span class="token punctuation">:</span></pre></td></tr><tr><td data-num="111"></td><td><pre>    <span class="token triple-quoted-string string">'''</span></pre></td></tr><tr><td data-num="112"></td><td><pre>    计算准确率</pre></td></tr><tr><td data-num="113"></td><td><pre>    传入:</pre></td></tr><tr><td data-num="114"></td><td><pre>        y        : 标签</pre></td></tr><tr><td data-num="115"></td><td><pre>        y_pred   : 预测值</pre></td></tr><tr><td data-num="116"></td><td><pre>    返回:</pre></td></tr><tr><td data-num="117"></td><td><pre>        accuracy : 准确率</pre></td></tr><tr><td data-num="118"></td><td><pre>    '''</pre></td></tr><tr><td data-num="119"></td><td><pre>    <span class="token keyword">return</span> np<span class="token punctuation">.</span>average<span class="token punctuation">(</span>y <span class="token operator">==</span> y_pred<span class="token punctuation">)</span><span class="token operator">*</span><span class="token number">100</span></pre></td></tr></table></figure><p>训练：</p><figure class="highlight python"><figcaption data-lang="python"></figcaption><table><tr><td data-num="1"></td><td><pre><span class="token comment"># 读取数据</span></pre></td></tr><tr><td data-num="2"></td><td><pre>iris_raw_data <span class="token operator">=</span> pd<span class="token punctuation">.</span>read_csv<span class="token punctuation">(</span><span class="token string">'iris.data'</span><span class="token punctuation">,</span> names  <span class="token operator">=</span><span class="token punctuation">[</span><span class="token string">'sepal length'</span><span class="token punctuation">,</span> <span class="token string">'sepal width'</span><span class="token punctuation">,</span> <span class="token string">'petal length'</span><span class="token punctuation">,</span> <span class="token string">'petal width'</span><span class="token punctuation">,</span> <span class="token string">'class'</span><span class="token punctuation">]</span><span class="token punctuation">)</span></pre></td></tr><tr><td data-num="3"></td><td><pre></pre></td></tr><tr><td data-num="4"></td><td><pre><span class="token comment"># 将三种类型映射成整数</span></pre></td></tr><tr><td data-num="5"></td><td><pre>Iris_dir <span class="token operator">=</span> <span class="token punctuation">&#123;</span><span class="token string">'Iris-setosa'</span><span class="token punctuation">:</span> <span class="token number">1</span><span class="token punctuation">,</span> <span class="token string">'Iris-versicolor'</span><span class="token punctuation">:</span> <span class="token number">2</span><span class="token punctuation">,</span> <span class="token string">'Iris-virginica'</span><span class="token punctuation">:</span> <span class="token number">3</span><span class="token punctuation">&#125;</span></pre></td></tr><tr><td data-num="6"></td><td><pre>iris_raw_data<span class="token punctuation">[</span><span class="token string">'class'</span><span class="token punctuation">]</span> <span class="token operator">=</span> iris_raw_data<span class="token punctuation">[</span><span class="token string">'class'</span><span class="token punctuation">]</span><span class="token punctuation">.</span><span class="token builtin">apply</span><span class="token punctuation">(</span><span class="token keyword">lambda</span> x<span class="token punctuation">:</span>Iris_dir<span class="token punctuation">[</span>x<span class="token punctuation">]</span><span class="token punctuation">)</span></pre></td></tr><tr><td data-num="7"></td><td><pre></pre></td></tr><tr><td data-num="8"></td><td><pre></pre></td></tr><tr><td data-num="9"></td><td><pre><span class="token comment"># 训练数据 X</span></pre></td></tr><tr><td data-num="10"></td><td><pre>iris_data <span class="token operator">=</span> iris_raw_data<span class="token punctuation">.</span>values<span class="token punctuation">[</span><span class="token punctuation">:</span><span class="token punctuation">,</span> <span class="token punctuation">:</span><span class="token operator">-</span><span class="token number">1</span><span class="token punctuation">]</span></pre></td></tr><tr><td data-num="11"></td><td><pre></pre></td></tr><tr><td data-num="12"></td><td><pre><span class="token comment"># 标签 y</span></pre></td></tr><tr><td data-num="13"></td><td><pre>y <span class="token operator">=</span> iris_raw_data<span class="token punctuation">.</span>values<span class="token punctuation">[</span><span class="token punctuation">:</span><span class="token punctuation">,</span> <span class="token operator">-</span><span class="token number">1</span><span class="token punctuation">]</span></pre></td></tr><tr><td data-num="14"></td><td><pre></pre></td></tr><tr><td data-num="15"></td><td><pre><span class="token comment"># 用 MGD 训练的参数</span></pre></td></tr><tr><td data-num="16"></td><td><pre>start <span class="token operator">=</span> time<span class="token punctuation">.</span>time<span class="token punctuation">(</span><span class="token punctuation">)</span></pre></td></tr><tr><td data-num="17"></td><td><pre>theta_MGD <span class="token operator">=</span> MGD_train<span class="token punctuation">(</span>iris_data<span class="token punctuation">,</span> y<span class="token punctuation">)</span></pre></td></tr><tr><td data-num="18"></td><td><pre>run_time <span class="token operator">=</span> time<span class="token punctuation">.</span>time<span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">-</span> start</pre></td></tr><tr><td data-num="19"></td><td><pre>y_pred_MGD <span class="token operator">=</span> GD_predict<span class="token punctuation">(</span>iris_data<span class="token punctuation">,</span> theta_MGD<span class="token punctuation">)</span></pre></td></tr><tr><td data-num="20"></td><td><pre><span class="token keyword">print</span><span class="token punctuation">(</span><span class="token string">"MGD训练1000轮得到的准确率&#123;:.2f&#125;% 运行时间是&#123;:.2f&#125;s"</span><span class="token punctuation">.</span><span class="token builtin">format</span><span class="token punctuation">(</span>calc_accuracy<span class="token punctuation">(</span>y<span class="token punctuation">,</span> y_pred_MGD<span class="token punctuation">)</span><span class="token punctuation">,</span> run_time<span class="token punctuation">)</span><span class="token punctuation">)</span></pre></td></tr><tr><td data-num="21"></td><td><pre></pre></td></tr><tr><td data-num="22"></td><td><pre><span class="token comment"># 用 SGD 训练的参数</span></pre></td></tr><tr><td data-num="23"></td><td><pre>start <span class="token operator">=</span> time<span class="token punctuation">.</span>time<span class="token punctuation">(</span><span class="token punctuation">)</span></pre></td></tr><tr><td data-num="24"></td><td><pre>theta_SGD <span class="token operator">=</span> SGD_train<span class="token punctuation">(</span>iris_data<span class="token punctuation">,</span> y<span class="token punctuation">)</span></pre></td></tr><tr><td data-num="25"></td><td><pre>run_time <span class="token operator">=</span> time<span class="token punctuation">.</span>time<span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">-</span> start</pre></td></tr><tr><td data-num="26"></td><td><pre>y_pred_SGD <span class="token operator">=</span> GD_predict<span class="token punctuation">(</span>iris_data<span class="token punctuation">,</span> theta_SGD<span class="token punctuation">)</span></pre></td></tr><tr><td data-num="27"></td><td><pre><span class="token keyword">print</span><span class="token punctuation">(</span><span class="token string">"SGD训练1000轮得到的准确率&#123;:.2f&#125;% 运行时间是&#123;:.2f&#125;s"</span><span class="token punctuation">.</span><span class="token builtin">format</span><span class="token punctuation">(</span>calc_accuracy<span class="token punctuation">(</span>y<span class="token punctuation">,</span> y_pred_SGD<span class="token punctuation">)</span><span class="token punctuation">,</span> run_time<span class="token punctuation">)</span><span class="token punctuation">)</span></pre></td></tr><tr><td data-num="28"></td><td><pre></pre></td></tr><tr><td data-num="29"></td><td><pre><span class="token comment"># 用 MBGD 训练的参数</span></pre></td></tr><tr><td data-num="30"></td><td><pre>start <span class="token operator">=</span> time<span class="token punctuation">.</span>time<span class="token punctuation">(</span><span class="token punctuation">)</span></pre></td></tr><tr><td data-num="31"></td><td><pre>theta_MBGD <span class="token operator">=</span> MBGD_train<span class="token punctuation">(</span>iris_data<span class="token punctuation">,</span> y<span class="token punctuation">)</span></pre></td></tr><tr><td data-num="32"></td><td><pre>run_time <span class="token operator">=</span> time<span class="token punctuation">.</span>time<span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">-</span> start</pre></td></tr><tr><td data-num="33"></td><td><pre>y_pred_MBGD <span class="token operator">=</span> GD_predict<span class="token punctuation">(</span>iris_data<span class="token punctuation">,</span> theta_MBGD<span class="token punctuation">)</span></pre></td></tr><tr><td data-num="34"></td><td><pre><span class="token keyword">print</span><span class="token punctuation">(</span><span class="token string">"MBGD训练1000轮得到的准确率&#123;:.2f&#125;% 运行时间是&#123;:.2f&#125;s"</span><span class="token punctuation">.</span><span class="token builtin">format</span><span class="token punctuation">(</span>calc_accuracy<span class="token punctuation">(</span>y<span class="token punctuation">,</span> y_pred_MBGD<span class="token punctuation">)</span><span class="token punctuation">,</span> run_time<span class="token punctuation">)</span><span class="token punctuation">)</span></pre></td></tr></table></figure><p>运行结果：</p><pre><code>MGD训练1000轮得到的准确率92.67% 运行时间是0.14s
SGD训练1000轮得到的准确率93.33% 运行时间是0.09s
MBGD训练1000轮得到的准确率92.67% 运行时间是0.14s
</code></pre><h1 id="6-三种数据集"><a class="anchor" href="#6-三种数据集">#</a> 6 三种数据集</h1><h2 id="61-训练集"><a class="anchor" href="#61-训练集">#</a> 6.1 训练集</h2><p>参与训练，模型从训练集中学习经验，从而不断减小训练误差。这个最容易理解，一般没什么疑惑。</p><h2 id="62-验证集"><a class="anchor" href="#62-验证集">#</a> 6.2 验证集</h2><p>不参与训练，用于在训练过程中检验模型的状态，收敛情况。验证集通常用于调整超参数，根据几组模型验证集上的表现决定哪组超参数拥有最好的性能。</p><p>同时验证集在训练过程中还可以用来监控模型是否发生过拟合，一般来说验证集表现稳定后，若继续训练，训练集表现还会继续上升，但是验证集会出现不升反降的情况，这样一般就发生了过拟合。所以验证集也用来判断何时停止训练。</p><h2 id="63-测试集"><a class="anchor" href="#63-测试集">#</a> 6.3 测试集</h2><p>不参与训练，用于在训练结束后对模型进行测试，评估其泛化能力。在之前模型使用【验证集】确定了【超参数】，使用【训练集】调整了【可训练参数】，最后使用一个从没有见过的数据集来判断这个模型的好坏。</p><p>三者区别<br>为了方便理解，人们常常把这三种数据集类比成学生的课本、作业和期末考：</p><ul><li>训练集 —— 课本，学生根据课本里的内容来掌握知识</li><li>验证集 —— 作业，通过作业可以知道不同学生实时的学习情况、进步的速度快慢</li><li>测试集 —— 考试，考的题是平常都没有见过，考察学生举一反三的能力</li></ul><p>传统上，一般三者切分的比例是：6：2：2，验证集并不是必须的。</p><h2 id="64-交叉验证"><a class="anchor" href="#64-交叉验证">#</a> 6.4 交叉验证</h2><p>假如我们教小朋友学加法：1 个苹果 + 1 个苹果 = 2 个苹果</p><p>当我们再测试的时候，会问：1 个香蕉 + 1 个香蕉 = 几个香蕉？</p><p>如果小朋友知道「2 个香蕉」，并且换成其他东西也没有问题，那么我们认为小朋友学习会了「1+1=2」这个知识点。</p><p>如果小朋友只知道「1 个苹果 + 1 个苹果 = 2 个苹果」，但是换成其他东西就不会了，那么我们就不能说小朋友学会了「1+1=2」这个知识点。</p><p><strong>评估模型是否学会了「某项技能」时，也需要用新的数据来评估，而不是用训练集里的数据来评估。这种「训练集」和「测试集」完全不同的验证方法就是交叉验证法。</strong></p><h1 id="7-回归模型评价指标"><a class="anchor" href="#7-回归模型评价指标">#</a> 7 回归模型评价指标</h1><p><img data-src="https://shun309.oss-cn-hangzhou.aliyuncs.com/photos/20210416093445.png" alt=""></p><p><img data-src="https://shun309.oss-cn-hangzhou.aliyuncs.com/photos/20210416093606.png" alt=""></p><p>参考：</p><p>1.<span class="exturl" data-url="aHR0cHM6Ly93d3cuY25ibG9ncy5jb20vZ2VvLXdpbGwvcC8xMDQ2ODI1My5odG1s">https://www.cnblogs.com/geo-will/p/10468253.html</span></p><p>2.<span class="exturl" data-url="aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80NDYxMzA2My9hcnRpY2xlL2RldGFpbHMvODg2NTk5ODE=">https://blog.csdn.net/weixin_44613063/article/details/88659981</span></p><p>3.<span class="exturl" data-url="aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L0hhb1ppSHVhbmcvYXJ0aWNsZS9kZXRhaWxzLzEwNDgxOTAyNg==">https://blog.csdn.net/HaoZiHuang/article/details/104819026</span></p><p>4.<span class="exturl" data-url="aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQzNjczMTE4L2FydGljbGUvZGV0YWlscy8xMDU0OTA1MDI=">https://blog.csdn.net/qq_43673118/article/details/105490502</span></p></div><footer><div class="meta"><span class="item"><span class="icon"><i class="ic i-calendar-check"></i> </span><span class="text">更新于</span> <time title="修改时间：2023-04-19 11:19:53" itemprop="dateModified" datetime="2023-04-19T11:19:53+08:00">2023-04-19</time> </span><span id="posts/b3064b8/" class="item leancloud_visitors" data-flag-title="梯度下降及线性回归详解" title="阅读次数"><span class="icon"><i class="ic i-eye"></i> </span><span class="text">阅读次数</span> <span class="leancloud-visitors-count"></span> <span class="text">次</span></span></div><div class="reward"><button><i class="ic i-heartbeat"></i> 赞赏</button><p>请我喝[茶]~(￣▽￣)~*</p><div id="qr"><div><img data-src="/images/wechatpay.png" alt="hang shun 微信支付"><p>微信支付</p></div><div><img data-src="/images/alipay.png" alt="hang shun 支付宝"><p>支付宝</p></div><div><img data-src="/images/paypal.png" alt="hang shun 贝宝"><p>贝宝</p></div></div></div><div id="copyright"><ul><li class="author"><strong>本文作者： </strong>hang shun <i class="ic i-at"><em>@</em></i>航 順</li><li class="link"><strong>本文链接：</strong> <a href="https://jiang-hs.gitee.io/posts/b3064b8/" title="梯度下降及线性回归详解">https://jiang-hs.gitee.io/posts/b3064b8/</a></li><li class="license"><strong>版权声明： </strong>本站所有文章除特别声明外，均采用 <span class="exturl" data-url="aHR0cHM6Ly9jcmVhdGl2ZWNvbW1vbnMub3JnL2xpY2Vuc2VzL2J5LW5jLXNhLzQuMC9kZWVkLnpo"><i class="ic i-creative-commons"><em>(CC)</em></i>BY-NC-SA</span> 许可协议。转载请注明出处！</li></ul></div></footer></article></div><div class="post-nav"><div class="item left"><a href="/posts/7c185f0e/" itemprop="url" rel="prev" data-background-image="https:&#x2F;&#x2F;pic1.imgdb.cn&#x2F;item&#x2F;64427c6f0d2dde5777affbe2.jpg" title="基于深度强化学习的长期推荐系统"><span class="type">上一篇</span> <span class="category"><i class="ic i-flag"></i></span><h3>基于深度强化学习的长期推荐系统</h3></a></div><div class="item right"><a href="/posts/ae4d52e6/" itemprop="url" rel="next" data-background-image="https:&#x2F;&#x2F;pic1.imgdb.cn&#x2F;item&#x2F;64427c720d2dde5777afff58.jpg" title="顺序价格机制的强化学习"><span class="type">下一篇</span> <span class="category"><i class="ic i-flag"></i></span><h3>顺序价格机制的强化学习</h3></a></div></div><div class="wrap" id="comments"></div></div><div id="sidebar"><div class="inner"><div class="panels"><div class="inner"><div class="contents panel pjax" data-title="文章目录"><ol class="toc"><li class="toc-item toc-level-1"><a class="toc-link" href="#1-%E4%B8%80%E5%85%83%E7%BA%BF%E6%80%A7%E5%9B%9E%E5%BD%92"><span class="toc-number">1.</span> <span class="toc-text">1 一元线性回归</span></a><ol class="toc-child"><li class="toc-item toc-level-2"><a class="toc-link" href="#11-%E4%BB%80%E4%B9%88%E6%98%AF%E5%9B%9E%E5%BD%92%E5%88%86%E6%9E%90"><span class="toc-number">1.1.</span> <span class="toc-text">1.1 什么是回归分析</span></a></li><li class="toc-item toc-level-2"><a class="toc-link" href="#12-%E7%BA%BF%E6%80%A7%E5%9B%9E%E5%BD%92"><span class="toc-number">1.2.</span> <span class="toc-text">1.2 线性回归</span></a></li><li class="toc-item toc-level-2"><a class="toc-link" href="#13-%E7%9B%AE%E6%A0%87%E6%8D%9F%E5%A4%B1%E5%87%BD%E6%95%B0"><span class="toc-number">1.3.</span> <span class="toc-text">1.3 目标 &#x2F; 损失函数</span></a></li><li class="toc-item toc-level-2"><a class="toc-link" href="#%E4%B8%A4%E7%A7%8D%E6%B1%82%E8%A7%A3%E6%96%B9%E5%BC%8F"><span class="toc-number">1.4.</span> <span class="toc-text">两种求解方式：</span></a></li></ol></li><li class="toc-item toc-level-1"><a class="toc-link" href="#2-%E6%A2%AF%E5%BA%A6%E4%B8%8B%E9%99%8Dgd"><span class="toc-number">2.</span> <span class="toc-text">2 梯度下降 (GD)</span></a><ol class="toc-child"><li class="toc-item toc-level-2"><a class="toc-link" href="#21-%E6%A2%AF%E5%BA%A6%E4%B8%8B%E9%99%8D%E7%AE%80%E4%BB%8B"><span class="toc-number">2.1.</span> <span class="toc-text">2.1 梯度下降简介</span></a></li><li class="toc-item toc-level-2"><a class="toc-link" href="#22-%E6%A0%B8%E5%BF%83%E5%85%AC%E5%BC%8F"><span class="toc-number">2.2.</span> <span class="toc-text">2.2 核心公式</span></a></li><li class="toc-item toc-level-2"><a class="toc-link" href="#23-%E6%A2%AF%E5%BA%A6%E4%B8%8B%E9%99%8D%E6%B3%95%E7%9A%84%E4%B8%80%E8%88%AC%E6%AD%A5%E9%AA%A4"><span class="toc-number">2.3.</span> <span class="toc-text">2.3 梯度下降法的一般步骤</span></a></li><li class="toc-item toc-level-2"><a class="toc-link" href="#24-%E4%B8%80%E5%85%83%E7%BA%BF%E6%80%A7%E5%9B%9E%E5%BD%92%E5%87%BD%E6%95%B0%E6%8E%A8%E5%AF%BC%E8%BF%87%E7%A8%8B"><span class="toc-number">2.4.</span> <span class="toc-text">2.4 一元线性回归函数推导过程</span></a></li><li class="toc-item toc-level-2"><a class="toc-link" href="#25-%E6%A2%AF%E5%BA%A6%E4%B8%8B%E9%99%8D%E5%AD%98%E5%9C%A8%E7%9A%84%E9%97%AE%E9%A2%98"><span class="toc-number">2.5.</span> <span class="toc-text">2.5 梯度下降存在的问题：</span></a></li><li class="toc-item toc-level-2"><a class="toc-link" href="#26-%E6%A2%AF%E5%BA%A6%E4%B8%8B%E9%99%8D%E4%B8%89%E5%85%84%E5%BC%9Fbgdsgdmbgd"><span class="toc-number">2.6.</span> <span class="toc-text">2.6 梯度下降三兄弟（BGD，SGD，MBGD）</span></a><ol class="toc-child"><li class="toc-item toc-level-3"><a class="toc-link" href="#261-%E6%89%B9%E9%87%8F%E6%A2%AF%E5%BA%A6%E4%B8%8B%E9%99%8D%E6%B3%95batch-gradient-descent"><span class="toc-number">2.6.1.</span> <span class="toc-text">2.6.1 批量梯度下降法（Batch Gradient Descent）</span></a></li><li class="toc-item toc-level-3"><a class="toc-link" href="#262-%E9%9A%8F%E6%9C%BA%E6%A2%AF%E5%BA%A6%E4%B8%8B%E9%99%8D%E6%B3%95stochastic-gradient-descent"><span class="toc-number">2.6.2.</span> <span class="toc-text">2.6.2 随机梯度下降法（Stochastic Gradient Descent）</span></a></li><li class="toc-item toc-level-3"><a class="toc-link" href="#3%E5%B0%8F%E6%89%B9%E9%87%8F%E6%A2%AF%E5%BA%A6%E4%B8%8B%E9%99%8D%E6%B3%95mini-batch-gradient-descent"><span class="toc-number">2.6.3.</span> <span class="toc-text">③小批量梯度下降法（Mini-batch Gradient Descent）</span></a></li></ol></li></ol></li><li class="toc-item toc-level-1"><a class="toc-link" href="#3-%E6%B3%A2%E5%A3%AB%E9%A1%BF%E6%88%BF%E4%BB%B7%E9%A2%84%E6%B5%8B"><span class="toc-number">3.</span> <span class="toc-text">3 波士顿房价预测</span></a><ol class="toc-child"><li class="toc-item toc-level-2"><a class="toc-link" href="#31-%E4%BB%A3%E7%A0%81"><span class="toc-number">3.1.</span> <span class="toc-text">3.1 代码</span></a></li><li class="toc-item toc-level-2"><a class="toc-link" href="#32-%E8%BE%93%E5%87%BA%E7%BB%93%E6%9E%9C"><span class="toc-number">3.2.</span> <span class="toc-text">3.2 输出结果：</span></a></li></ol></li><li class="toc-item toc-level-1"><a class="toc-link" href="#4-%E5%A4%9A%E5%85%83%E7%BA%BF%E6%80%A7%E5%9B%9E%E5%BD%92%E5%8F%8A%E6%A2%AF%E5%BA%A6%E4%B8%8B%E9%99%8D"><span class="toc-number">4.</span> <span class="toc-text">4 多元线性回归及梯度下降</span></a><ol class="toc-child"><li class="toc-item toc-level-2"><a class="toc-link" href="#41-%E5%AE%9A%E4%B9%89%E6%95%B0%E6%8D%AE"><span class="toc-number">4.1.</span> <span class="toc-text">4.1 定义数据</span></a></li><li class="toc-item toc-level-2"><a class="toc-link" href="#42-%E5%AE%9A%E4%B9%89%E5%87%BD%E6%95%B0"><span class="toc-number">4.2.</span> <span class="toc-text">4.2 定义函数</span></a></li><li class="toc-item toc-level-2"><a class="toc-link" href="#43-%E6%A2%AF%E5%BA%A6%E4%B8%8B%E9%99%8D"><span class="toc-number">4.3.</span> <span class="toc-text">4.3 梯度下降</span></a></li></ol></li><li class="toc-item toc-level-1"><a class="toc-link" href="#5-%E9%B8%A2%E5%B0%BE%E8%8A%B1%E6%95%B0%E6%8D%AE%E9%9B%86"><span class="toc-number">5.</span> <span class="toc-text">5 鸢尾花数据集</span></a></li><li class="toc-item toc-level-1"><a class="toc-link" href="#6-%E4%B8%89%E7%A7%8D%E6%95%B0%E6%8D%AE%E9%9B%86"><span class="toc-number">6.</span> <span class="toc-text">6 三种数据集</span></a><ol class="toc-child"><li class="toc-item toc-level-2"><a class="toc-link" href="#61-%E8%AE%AD%E7%BB%83%E9%9B%86"><span class="toc-number">6.1.</span> <span class="toc-text">6.1 训练集</span></a></li><li class="toc-item toc-level-2"><a class="toc-link" href="#62-%E9%AA%8C%E8%AF%81%E9%9B%86"><span class="toc-number">6.2.</span> <span class="toc-text">6.2 验证集</span></a></li><li class="toc-item toc-level-2"><a class="toc-link" href="#63-%E6%B5%8B%E8%AF%95%E9%9B%86"><span class="toc-number">6.3.</span> <span class="toc-text">6.3 测试集</span></a></li><li class="toc-item toc-level-2"><a class="toc-link" href="#64-%E4%BA%A4%E5%8F%89%E9%AA%8C%E8%AF%81"><span class="toc-number">6.4.</span> <span class="toc-text">6.4 交叉验证</span></a></li></ol></li><li class="toc-item toc-level-1"><a class="toc-link" href="#7-%E5%9B%9E%E5%BD%92%E6%A8%A1%E5%9E%8B%E8%AF%84%E4%BB%B7%E6%8C%87%E6%A0%87"><span class="toc-number">7.</span> <span class="toc-text">7 回归模型评价指标</span></a></li></ol></div><div class="related panel pjax" data-title="系列文章"></div><div class="overview panel" data-title="站点概览"><div class="author" itemprop="author" itemscope itemtype="http://schema.org/Person"><img class="image" itemprop="image" alt="hang shun" data-src="/images/avatar.jpg"><p class="name" itemprop="name">hang shun</p><div class="description" itemprop="description">世中逢尔，雨中逢花</div></div><nav class="state"><div class="item posts"><a href="/archives/"><span class="count">45</span> <span class="name">文章</span></a></div><div class="item categories"><a href="/categories/"><span class="count">10</span> <span class="name">分类</span></a></div><div class="item tags"><a href="/tags/"><span class="count">25</span> <span class="name">标签</span></a></div></nav><div class="social"><span class="exturl item github" data-url="aHR0cHM6Ly9naXRodWIuY29tL0pJQU5HLUhT" title="https:&#x2F;&#x2F;github.com&#x2F;JIANG-HS"><i class="ic i-github"></i></span> <span class="exturl item zhihu" data-url="aHR0cHM6Ly93d3cuemhpaHUuY29tL3Blb3BsZS9odWktc2h1bi14aW4tbGl1" title="https:&#x2F;&#x2F;www.zhihu.com&#x2F;people&#x2F;hui-shun-xin-liu"><i class="ic i-zhihu"></i></span> <span class="exturl item music" data-url="aHR0cHM6Ly9tdXNpYy4xNjMuY29tLyMvdXNlci9ob21lP2lkPTE4MzkwMTczMzI=" title="https:&#x2F;&#x2F;music.163.com&#x2F;#&#x2F;user&#x2F;home?id&#x3D;1839017332"><i class="ic i-cloud-music"></i></span> <span class="exturl item bilibili" data-url="aHR0cHM6Ly9zcGFjZS5iaWxpYmlsaS5jb20vMzIxMTYyNDg1" title="https:&#x2F;&#x2F;space.bilibili.com&#x2F;321162485"><i class="ic i-bilibili"></i></span></div><ul class="menu"><li class="item"><a href="/" rel="section"><i class="ic i-home"></i>首页</a></li><li class="item"><a href="/about/" rel="section"><i class="ic i-user"></i>关于</a></li><li class="item dropdown"><a href="javascript:void(0);"><i class="ic i-feather"></i>文章</a><ul class="submenu"><li class="item"><a href="/archives/" rel="section"><i class="ic i-list-alt"></i>归档</a></li><li class="item"><a href="/categories/" rel="section"><i class="ic i-th"></i>分类</a></li><li class="item"><a href="/tags/" rel="section"><i class="ic i-tags"></i>标签</a></li></ul></li><li class="item"><a href="/friends/" rel="section"><i class="ic i-heart"></i>友達</a></li><li class="item"><a href="/movie/" rel="section"><i class="ic i-play"></i>movie</a></li><li class="item"><a href="/music/" rel="section"><i class="ic i-music"></i>music</a></li></ul></div></div></div><ul id="quick"><li class="prev pjax"><a href="/posts/7c185f0e/" rel="prev" title="上一篇"><i class="ic i-chevron-left"></i></a></li><li class="up"><i class="ic i-arrow-up"></i></li><li class="down"><i class="ic i-arrow-down"></i></li><li class="next pjax"><a href="/posts/ae4d52e6/" rel="next" title="下一篇"><i class="ic i-chevron-right"></i></a></li><li class="percent"></li></ul></div></div><div class="dimmer"></div></div></main><footer id="footer"><div class="inner"><div class="widgets"><div class="rpost pjax"><h2>随机文章</h2><ul><li class="item"><div class="breadcrumb"></div><span><a href="/posts/7c4ca347/" title="计算机网络概述">计算机网络概述</a></span></li><li class="item"><div class="breadcrumb"><a href="/categories/%E6%9C%BA%E5%99%A8%E5%AD%A6%E4%B9%A0%E5%9F%BA%E7%A1%80/" title="分类于 机器学习基础">机器学习基础</a></div><span><a href="/posts/20417848/" title="循环神经网络">循环神经网络</a></span></li><li class="item"><div class="breadcrumb"><a href="/categories/%E6%9C%BA%E5%99%A8%E5%AD%A6%E4%B9%A0%E5%9F%BA%E7%A1%80/" title="分类于 机器学习基础">机器学习基础</a></div><span><a href="/posts/a10feb4a/" title="协同过滤算法">协同过滤算法</a></span></li><li class="item"><div class="breadcrumb"></div><span><a href="/posts/67decac2/" title="初识GCN">初识GCN</a></span></li><li class="item"><div class="breadcrumb"><a href="/categories/%E6%9C%BA%E5%99%A8%E5%AD%A6%E4%B9%A0%E5%9F%BA%E7%A1%80/" title="分类于 机器学习基础">机器学习基础</a></div><span><a href="/posts/fe5ae0e7/" title="基于矩阵分解的推荐算法">基于矩阵分解的推荐算法</a></span></li><li class="item"><div class="breadcrumb"></div><span><a href="/posts/6a16b017/" title="51单片机基础-4">51单片机基础-4</a></span></li><li class="item"><div class="breadcrumb"><a href="/categories/%E8%AE%BA%E6%96%87%E7%B2%BE%E8%AF%BB/" title="分类于 论文精读">论文精读</a> <i class="ic i-angle-right"></i> <a href="/categories/%E8%AE%BA%E6%96%87%E7%B2%BE%E8%AF%BB/%E6%96%B0%E9%97%BB%E6%8E%A8%E8%8D%90/" title="分类于 新闻推荐">新闻推荐</a> <i class="ic i-angle-right"></i> <a href="/categories/%E8%AE%BA%E6%96%87%E7%B2%BE%E8%AF%BB/%E6%96%B0%E9%97%BB%E6%8E%A8%E8%8D%90/%E9%9A%90%E7%A7%81%E4%BF%9D%E6%8A%A4%E6%96%B0%E9%97%BB%E6%8E%A8%E8%8D%90/" title="分类于 隐私保护新闻推荐">隐私保护新闻推荐</a> <i class="ic i-angle-right"></i> <a href="/categories/%E8%81%94%E9%82%A6%E5%AD%A6%E4%B9%A0/" title="分类于 联邦学习">联邦学习</a></div><span><a href="/posts/ef078385/" title="FedNewsRec：隐私保护新闻推荐模型学习">FedNewsRec：隐私保护新闻推荐模型学习</a></span></li><li class="item"><div class="breadcrumb"></div><span><a href="/posts/9dc5ce37/" title="DIB-PEB-Sequientia-RS笔记">DIB-PEB-Sequientia-RS笔记</a></span></li><li class="item"><div class="breadcrumb"></div><span><a href="/posts/7c185f0e/" title="基于深度强化学习的长期推荐系统">基于深度强化学习的长期推荐系统</a></span></li><li class="item"><div class="breadcrumb"><a href="/categories/%E8%AE%BA%E6%96%87%E7%B2%BE%E8%AF%BB/" title="分类于 论文精读">论文精读</a> <i class="ic i-angle-right"></i> <a href="/categories/%E8%AE%BA%E6%96%87%E7%B2%BE%E8%AF%BB/%E6%96%B0%E9%97%BB%E6%8E%A8%E8%8D%90/" title="分类于 新闻推荐">新闻推荐</a> <i class="ic i-angle-right"></i> <a href="/categories/%E8%AE%BA%E6%96%87%E7%B2%BE%E8%AF%BB/%E6%96%B0%E9%97%BB%E6%8E%A8%E8%8D%90/%E9%9A%90%E7%A7%81%E4%BF%9D%E6%8A%A4%E6%96%B0%E9%97%BB%E6%8E%A8%E8%8D%90/" title="分类于 隐私保护新闻推荐">隐私保护新闻推荐</a></div><span><a href="/posts/b2c2f458/" title="Adv-MultVAE：基于对抗学习的隐私保护推荐算法">Adv-MultVAE：基于对抗学习的隐私保护推荐算法</a></span></li></ul></div><div><h2>最新评论</h2><ul class="leancloud-recent-comment"></ul></div></div><div class="status"><div class="copyright">&copy; 2020 – <span itemprop="copyrightYear">2023</span> <span class="with-love"><i class="ic i-sakura rotate"></i> </span><span class="author" itemprop="copyrightHolder">hang shun @ hang shun</span></div><div class="count"><span class="post-meta-item-icon"><i class="ic i-chart-area"></i> </span><span title="站点总字数">267k 字</span> <span class="post-meta-divider">|</span> <span class="post-meta-item-icon"><i class="ic i-coffee"></i> </span><span title="站点阅读时长">4:02</span></div><div class="powered-by">基于 <span class="exturl" data-url="aHR0cHM6Ly9oZXhvLmlv">Hexo</span> & Theme.<span class="exturl" data-url="aHR0cHM6Ly9naXRodWIuY29tL2FtZWhpbWUvaGV4by10aGVtZS1zaG9rYQ==">Shoka</span></div></div></div></footer></div><script data-config type="text/javascript">var LOCAL={path:"posts/b3064b8/",favicon:{show:"(´Д｀)被发现了！",hide:"（●´3｀●）我藏好了~"},search:{placeholder:"文章搜索",empty:"关于 「 ${query} 」，什么也没搜到",stats:"${time} ms 内找到 ${hits} 条结果"},valine:!0,copy_tex:!0,katex:!0,fancybox:!0,copyright:'复制成功，转载请遵守 <i class="ic i-creative-commons"></i>BY-NC-SA 协议。',ignores:[function(e){return e.includes("#")},function(e){return new RegExp(LOCAL.path+"$").test(e)}]}</script><script src="https://cdn.polyfill.io/v2/polyfill.js"></script><script src="//cdn.jsdelivr.net/combine/npm/pace-js@1.0.2/pace.min.js,npm/pjax@0.2.8/pjax.min.js,npm/whatwg-fetch@3.4.0/dist/fetch.umd.min.js,npm/animejs@3.2.0/lib/anime.min.js,npm/algoliasearch@4/dist/algoliasearch-lite.umd.js,npm/instantsearch.js@4/dist/instantsearch.production.min.js,npm/lozad@1/dist/lozad.min.js,npm/quicklink@2/dist/quicklink.umd.js"></script><script src="/js/app.js?v=0.0.0"></script></body></html><!-- rebuild by hrmmi -->